Re: [gmx-users] GPU-gromacs

[Carsten Kutzner]

On Oct 25, 2013, at 4:07 PM, aixintiankong  wrote:

> Dear prof.,
> i want install gromacs on a multi-core workstation with a GPU(tesla c2075), 
> should i install the openmpi or mpich2? 
If you want to run Gromacs on just one workstation with a single GPU, you do
not need to install an MPI library at all!

Carsten

> -- 
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the 
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists


--
Dr. Carsten Kutzner
Max Planck Institute for Biophysical Chemistry
Theoretical and Computational Biophysics
Am Fassberg 11, 37077 Goettingen, Germany
Tel. +49-551-2012313, Fax: +49-551-2012302
http://www.mpibpc.mpg.de/grubmueller/kutzner
http://www.mpibpc.mpg.de/grubmueller/sppexa

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU-gromacs

[aixintiankong]

Dear prof.,
i want install gromacs on a multi-core workstation with a GPU(tesla c2075), 
should i install the openmpi or mpich2? 
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of Gromacs

[Justin Lemkul]

On 8/19/13 5:38 AM, grita wrote:

Hey guys,

Is it possible to make a SD simulation with using the pull code in the GPU
version of Gromacs?



Have you tried it?

-Justin

--
==

Justin A. Lemkul, Ph.D.
Postdoctoral Fellow

Department of Pharmaceutical Sciences
School of Pharmacy
Health Sciences Facility II, Room 601
University of Maryland, Baltimore
20 Penn St.
Baltimore, MD 21201

jalem...@outerbanks.umaryland.edu | (410) 706-7441

==
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU version of Gromacs

[grita]

Hey guys,

Is it possible to make a SD simulation with using the pull code in the GPU
version of Gromacs?

Best, grita



--
View this message in context: 
http://gromacs.5086.x6.nabble.com/GPU-version-of-Gromacs-tp5010581.html
Sent from the GROMACS Users Forum mailing list archive at Nabble.com.
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU metadynamics

[Albert]

On 08/15/2013 11:21 AM, Jacopo Sgrignani wrote:

Dear Albert
to run parallel jobs on multiple GPUs you should use something like this:

mpirun -np (number of parallel sessions on CPU) mdrun_mpi .. -gpu_id 


so you will have 4 calculations for GPU.


Jacopo


thanks a  lot for reply. but there is some problem with following command:

mpirun -np 4 mdrun_mpi -s md.tpr -v -g md.log -o md.trr -x md.xtc 
-plumed plumed2.dat -e md.edr -gpu_id 0123.


--log---

4 GPUs detected on host node3:
  #0: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #3: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible


---
Program mdrun_mpi, VERSION 4.6.3
Source code file: 
/home/albert/install/source/gromacs-4.6.3/src/gmxlib/gmx_detect_hardware.c, 
line: 349


Fatal error:
Incorrect launch configuration: mismatching number of PP MPI processes 
and GPUs per node.
mdrun_mpi was started with 1 PP MPI process per node, but you provided 4 
GPUs.

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
---

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU metadynamics

[Jacopo Sgrignani]

Dear Albert
to run parallel jobs on multiple GPUs you should use something like this:

mpirun -np (number of parallel sessions on CPU) mdrun_mpi .. -gpu_id 


so you will have 4 calculations for GPU.


Jacopo

Inviato da iPad

Il giorno 15/ago/2013, alle ore 10:56, Albert  ha scritto:

> Hello:
> 
> I've got two GTX690 GPU in a workstation, and I compiled gromacs-4.6.3 with 
> plumed and MPI support. I am trying to run some metadynamics with mdrun with 
> command:
> 
> mdrun_mpi -s md.tpr -v -g md.log -o md.trr -x md.xtc -plumed plumed2.dat -e 
> md.edr
> 
> but mdrun can only use 1 GPU as indicated in the log file:
> 
> 
> 
> 4 GPUs detected on host node3:
>  #0: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>  #1: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>  #2: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>  #3: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
> 
> 
> NOTE: potentially sub-optimal launch configuration, mdrun_mpi started with 
> less
>  PP MPI process per node than GPUs available.
>  Each PP MPI process can use only one GPU, 1 GPUs per node will be used.
> 
> 1 GPU auto-selected for this run: #0
> 
> 
> 
> I am just wondering how can we use multiple GPU for such kind of jobs?
> 
> THX
> Albert
> -- 
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www interface 
> or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU metadynamics

[Albert]

Hello:

 I've got two GTX690 GPU in a workstation, and I compiled gromacs-4.6.3 
with plumed and MPI support. I am trying to run some metadynamics with 
mdrun with command:


 mdrun_mpi -s md.tpr -v -g md.log -o md.trr -x md.xtc -plumed 
plumed2.dat -e md.edr


but mdrun can only use 1 GPU as indicated in the log file:



4 GPUs detected on host node3:
  #0: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #3: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible


NOTE: potentially sub-optimal launch configuration, mdrun_mpi started 
with less

  PP MPI process per node than GPUs available.
  Each PP MPI process can use only one GPU, 1 GPUs per node will be 
used.


1 GPU auto-selected for this run: #0



I am just wondering how can we use multiple GPU for such kind of jobs?

THX
Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU + surface

[Lucio Montero]

Hello. ¿Have you removed periodicity?. Because you may only be seeing 
traversal of water molecules among copies of the periodic system.

Lucio Montero
Ph. D. student
Instituto de Biotecnologia, UNAM
Mexico


El 08/08/13 07:39, Ondrej Kroutil escribió:

Dear GMX users.
   I have done simulation of ions and water near quartz surface
(ClayFF) using GPU (GTX580) and Gromacs (4.6.1, single precision, 64
bit, SSE4.1, fftw-3.3.3) and have observed strange behavior of water
and ions. Its NVT simulation with freezed surface atoms (see .mdp
below) and negative charge on surface (deprotonated silanols), system
is overall neutral. I used same mdp for normal CPU simulation and GPU
simulation, and just added -testverlet option for GPU simulation.
   In CPU simulation ions and water behaved as expected (see
http://i1315.photobucket.com/albums/t587/Andrew_Twister/cpu-simul_zpscf784b46.png)
, but in GPU simulation there was a visible flow of ions toward image
of lower surface and all water molecules were oriented with hydrogens
facing downward and oxygens oriented upwards (see
http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-simul_zps2c160ea6.png).
It looks like there was an applied electric field but it is not.
   Do you think there is a problem in initial setup of parameters in
mdp file? Or maybe problem of freezing groups? With no freeze
situation is better, but there is still visible flow and pairing of
same ions (see 
http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-no_freeze_zps72ef3938.png).
   It look as electrostatics problem. Do you have any hints, please?
And sorry if I missed similar topic in mailing list, but I couldn't
find anything similar.

   Ondrej Kroutil

integrator   =  md
dt   =  0.001
nsteps   =  10
comm_mode=  linear
nstcomm  =  1000
nstxout  =  0
nstxtcout=  1000
nstvout  =  0
nstfout  =  0
nstlog   =  1000
xtc_precision=  1
nstlist  =  10
ns_type  =  grid
rlist=  1.2
coulombtype  =  PME
rcoulomb =  1.2
rvdw =  1.2
constraints  =  hbonds
constraint_algorithm =  lincs
lincs_iter   =  1
fourierspacing   =  0.1
pme_order   =  4
ewald_rtol  =  1e-5
ewald_geometry  =  3dc
optimize_fft=  yes
; Nose-Hoover temperature coupling
Tcoupl =  nose-hoover
tau_t  =  1
tc_grps=  system
ref_t  =  298.15
; No Pressure
; Pcoupl =   Parrinello-Rahman
pcoupltype  =  semiisotropic
tau_p   =  1.0
compressibility =  0 4.6e-5
ref_p   =  0 1.0
; OTHER
periodic_molecules  =  no
pbc =  xyz
;energygrps = SOL SOH
freezegrps  = BULK
freezedim   = Y Y Y
gen_vel = yes
gen_temp= 298.15
gen_seed= -1



--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

RE: [gmx-users] GPU + surface

[Berk Hess]

Hi,

The -testverlet option is only for testing (as the name implies).
Please set the mdp option cutoff-scheme = Verlet
Also please update to 4.6.3, as this, potential, issue might have already been 
resolved.
With the Verlet scheme the CPU and GPU should give the same, correct or 
incorrect, result.

Could it be that your system is located partially above and partially below z=0?
This will cause problems with ewald-geometry = 3dc. To use this option you need 
to ensure your whole system is in the same periodic image.

Cheers,

Berk

---
> Date: Thu, 8 Aug 2013 14:39:59 +0200
> From: okrou...@gmail.com
> To: gmx-users@gromacs.org
> Subject: [gmx-users] GPU + surface
>
> Dear GMX users.
> I have done simulation of ions and water near quartz surface
> (ClayFF) using GPU (GTX580) and Gromacs (4.6.1, single precision, 64
> bit, SSE4.1, fftw-3.3.3) and have observed strange behavior of water
> and ions. Its NVT simulation with freezed surface atoms (see .mdp
> below) and negative charge on surface (deprotonated silanols), system
> is overall neutral. I used same mdp for normal CPU simulation and GPU
> simulation, and just added -testverlet option for GPU simulation.
> In CPU simulation ions and water behaved as expected (see
> http://i1315.photobucket.com/albums/t587/Andrew_Twister/cpu-simul_zpscf784b46.png)
> , but in GPU simulation there was a visible flow of ions toward image
> of lower surface and all water molecules were oriented with hydrogens
> facing downward and oxygens oriented upwards (see
> http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-simul_zps2c160ea6.png).
> It looks like there was an applied electric field but it is not.
> Do you think there is a problem in initial setup of parameters in
> mdp file? Or maybe problem of freezing groups? With no freeze
> situation is better, but there is still visible flow and pairing of
> same ions (see 
> http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-no_freeze_zps72ef3938.png).
> It look as electrostatics problem. Do you have any hints, please?
> And sorry if I missed similar topic in mailing list, but I couldn't
> find anything similar.
>
> Ondrej Kroutil
>
> integrator = md
> dt = 0.001
> nsteps = 10
> comm_mode = linear
> nstcomm = 1000
> nstxout = 0
> nstxtcout = 1000
> nstvout = 0
> nstfout = 0
> nstlog = 1000
> xtc_precision = 1
> nstlist = 10
> ns_type = grid
> rlist = 1.2
> coulombtype = PME
> rcoulomb = 1.2
> rvdw = 1.2
> constraints = hbonds
> constraint_algorithm = lincs
> lincs_iter = 1
> fourierspacing = 0.1
> pme_order = 4
> ewald_rtol = 1e-5
> ewald_geometry = 3dc
> optimize_fft = yes
> ; Nose-Hoover temperature coupling
> Tcoupl = nose-hoover
> tau_t = 1
> tc_grps = system
> ref_t = 298.15
> ; No Pressure
> ; Pcoupl = Parrinello-Rahman
> pcoupltype = semiisotropic
> tau_p = 1.0
> compressibility = 0 4.6e-5
> ref_p = 0 1.0
> ; OTHER
> periodic_molecules = no
> pbc = xyz
> ;energygrps = SOL SOH
> freezegrps = BULK
> freezedim = Y Y Y
> gen_vel = yes
> gen_temp = 298.15
> gen_seed = -1
>
> --
> Ondřej Kroutil
> ,, Faculty of Health and Social Studies
> "))' University of South Bohemia
> OOO Jirovcova 24, Ceske Budejovice
> OOO The Czech Republic
> | OO E-mail: okrou...@gmail.com
>>-- O Mobile: +420 736 537 190
> --
> gmx-users mailing list gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists   
>   --
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU + surface

[Ondrej Kroutil]

Dear GMX users.
  I have done simulation of ions and water near quartz surface
(ClayFF) using GPU (GTX580) and Gromacs (4.6.1, single precision, 64
bit, SSE4.1, fftw-3.3.3) and have observed strange behavior of water
and ions. Its NVT simulation with freezed surface atoms (see .mdp
below) and negative charge on surface (deprotonated silanols), system
is overall neutral. I used same mdp for normal CPU simulation and GPU
simulation, and just added -testverlet option for GPU simulation.
  In CPU simulation ions and water behaved as expected (see
http://i1315.photobucket.com/albums/t587/Andrew_Twister/cpu-simul_zpscf784b46.png)
, but in GPU simulation there was a visible flow of ions toward image
of lower surface and all water molecules were oriented with hydrogens
facing downward and oxygens oriented upwards (see
http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-simul_zps2c160ea6.png).
It looks like there was an applied electric field but it is not.
  Do you think there is a problem in initial setup of parameters in
mdp file? Or maybe problem of freezing groups? With no freeze
situation is better, but there is still visible flow and pairing of
same ions (see 
http://i1315.photobucket.com/albums/t587/Andrew_Twister/gpu-no_freeze_zps72ef3938.png).
  It look as electrostatics problem. Do you have any hints, please?
And sorry if I missed similar topic in mailing list, but I couldn't
find anything similar.

  Ondrej Kroutil

integrator   =  md
dt   =  0.001
nsteps   =  10
comm_mode=  linear
nstcomm  =  1000
nstxout  =  0
nstxtcout=  1000
nstvout  =  0
nstfout  =  0
nstlog   =  1000
xtc_precision=  1
nstlist  =  10
ns_type  =  grid
rlist=  1.2
coulombtype  =  PME
rcoulomb =  1.2
rvdw =  1.2
constraints  =  hbonds
constraint_algorithm =  lincs
lincs_iter   =  1
fourierspacing   =  0.1
pme_order   =  4
ewald_rtol  =  1e-5
ewald_geometry  =  3dc
optimize_fft=  yes
; Nose-Hoover temperature coupling
Tcoupl =  nose-hoover
tau_t  =  1
tc_grps=  system
ref_t  =  298.15
; No Pressure
; Pcoupl =   Parrinello-Rahman
pcoupltype  =  semiisotropic
tau_p   =  1.0
compressibility =  0 4.6e-5
ref_p   =  0 1.0
; OTHER
periodic_molecules  =  no
pbc =  xyz
;energygrps = SOL SOH
freezegrps  = BULK
freezedim   = Y Y Y
gen_vel = yes
gen_temp= 298.15
gen_seed= -1

-- 
Ondřej Kroutil
,,  Faculty of Health and Social Studies
   "))' University of South Bohemia
 OOO   Jirovcova 24, Ceske Budejovice
  OOO  The Czech Republic
| OO E-mail:  okrou...@gmail.com
>--O Mobile:  +420 736 537 190
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: Re: Re: [gmx-users] GPU-based workstation

[Szilárd Páll]

I may be late with the reply, but here are my 2 cents.

If you need a single very fast machine (i.e. maximum single simulation
performance), you should get
- either a very fast desktop CPU: i7 3930 or for 2x more the 3970 -
which, BTW, I think is not worth it ($600-1000)
- or 1-2 fast Xeon E5-s - depending on how many and which these will
be $1k-2k each.

For a single CPU setup two Titans may be an overkill and (at least
with the current code) you may get very little extra performance from
using two iso one GPU. With a dual-socket machine (and decently fast
CPUs), if you have a large enough input system, two GPUs will work
nicely.

However, if you care about total simulation throughput and you have
multiple simulations to run, I'd suggest that you buy 2-3 machines
with the components that give the best ns/day/$: something like
i7-4670 or 4770 with GTX 680/770 (or 780).



--
Szilárd


On Thu, Jun 27, 2013 at 1:01 PM, James Starlight  wrote:
> Back to my question
> I want to build gpu-based workstation based onto 2 titans geforces.
>
> My current budget allow me only hight-end  6nodes core i 7-3930  and MB
> with 5 PCI-E (like Asus rampage IV series). Would this system be balanced
> with two GPUs ? Should I use two 6-8 nodes XEONS instead of i7?
>
> James
>
> 2013/5/29 James Starlight 
>
>> Dear Dr. Pall!
>>
>> Thank you for your suggestions!
>>
>> Asumming that I have budget of 5000 $ and I want to build gpu-based
>> desktop on this money.
>>
>> Previously I've used single 4 core i5 with GTX 670 and obtain average 10
>> ns\day performance for the 70k atoms systems (1.0 cutoffs, no virtual sites
>> , sd integrator).
>>
>> Now I'd like to build system based on 2 hight-end GeForces (e.g like
>> TITAN).
>> Should that system include 2 cpu's for good balancing? (e.g two 6 nodes
>> XEONS with faster clocks for instance could be better for simulations than
>> i7, couldnt it?)
>>
>> What addition properties to the MB should I consider for such system ?
>>
>> James
>>
>>
>> 2013/5/28 lloyd riggs 
>>
>>> Dear Dr. Pali,
>>>
>>> Thank you,
>>>
>>> Stephan Watkins
>>>
>>> *Gesendet:* Dienstag, 28. Mai 2013 um 19:50 Uhr
>>> *Von:* "Szilárd Páll" 
>>>
>>> *An:* "Discussion list for GROMACS users" 
>>> *Betreff:* Re: Re: [gmx-users] GPU-based workstation
>>> Dear all,
>>>
>>> As far as I understand, the OP is interested in hardware for *running*
>>> GROMACS 4.6 rather than developing code. or running LINPACK.
>>>
>>>
>>> To get best performance it is important to use a machine with hardware
>>> balanced for GROMACS' workloads. Too little GPU resources will result
>>> in CPU idling; too much GPU resources will lead to the runs being CPU
>>> or multi-GPU scaling bound and above a certain level GROMACS won't be
>>> able to make use of additional GPUs.
>>>
>>> Of course, the balance will depend both on hardware and simulation
>>> settings (mostly the LJ cut-off used).
>>>
>>> An additional factor to consider is typical system size. To reach near
>>> peak pair-force throughput on GPUs you typically need >20k-40k
>>> particles/GPU (depends on the architecture) and throughput drops below
>>> these values. Hence, in most cases it is preferred to use fewer and
>>> faster GPUs rather than more.
>>>
>>> Without knowing the budgdet and indented use of the machine it is hard
>>> to make suggestions, but I would say for a budget desktop box a
>>> quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
>>> fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
>>> If you're considering dual-socket workstations, I suggest you go with
>>> the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
>>> GHz), otherwise you may not see as much benefit as you would expect
>>> based on the insane price tag (especially if you compare to an i7
>>> 3939K or its IVB successor).
>>>
>>> Cheers,
>>> --
>>> Szilárd
>>>
>>>
>>> On Sat, May 25, 2013 at 1:02 PM, lloyd riggs  wrote:
>>> > More RAM the better, and the best I have seen is 4 GPU work station. I
>>> can
>>> > use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
>>> > really 3-4 GPU, except the tyan mentioned (there designed as blades so
>>> an 8
>>> > or 10 slot board really holds 8 or 10 GPU's). There's co

Re: [gmx-users] gpu cluster explanation

[Francesco]

Hi Richard,
Thank you for the help and sorry for the delay in my reply.
I tried some test run changing some parameters (e.g. removing PME) and I
was able to reach 20ns/day, so I think that 9-11 ns/day it's the max
that I can obtain for my setting.

thank your again for your help.

cheers,

Fra

On Fri, 12 Jul 2013, at 03:41 PM, Richard Broadbent wrote:
> 
> 
> On 12/07/13 13:26, Francesco wrote:
> > Hi all,
> > I'm working with a 200K atoms system (protein + explicit water) and
> > after a while using a cpu cluster I had to switch to a gpu cluster.
> > I read both Acceleration and parallelization and Gromacs-gpu
> > documentation pages
> > (http://www.gromacs.org/Documentation/Acceleration_and_parallelization
> > and
> > http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GROMACS-OpenMM)
> > but it's a bit confusing and I need help to understand if I really have
> > understood correctly. :)
> > I have 2 type of nodes:
> > 3gpu ( NVIDIA Tesla M2090) and 2 cpu 6cores each (Intel Xeon E5649 @
> > 2.53GHz)
> > 8gpu and 2 cpu (6 cores each)
> >
> > 1) I can only have 1 MPI per gpu, meaning that with 3 gpu I can have 3
> > MPI max.
> > 2) because I have 12 cores I can open 4 OPenMP threads x MPI, because
> > 4x3= 12
> >
> > now if I have a node with 8 gpu, I can use 4 gpu:
> > 4 MPI and 3 OpenMP
> > is it right?
> > is it possible to use 8 gpu and 8 cores only?
> 
> you could set -ntomp 0, however and setup mpi/thread mpi to use 8 cores. 
> However, a system that unbalanced (huge amount of gpu power to 
> comparatively little cpu power) is unlikely to get great performance.
> >
> > Using gromacs 4.6.2 and 144 cpu cores I reach 35 ns/day, while with 3
> > gpu  and 12 cores I get 9-11 ns/day.
> >
> That slowdown is in line with what I got when I tried a similar cpu-gpu 
> setup. That said other's might have some advice that will improve your 
> performance.
> 
> > the command that I use is:
> > mdrun -dlb yes -s input_50.tpr -deffnm 306s_50 -v
> > with n° gpu set via script :
> > #BSUB -n 3
> >
> > I also tried to set -npme / -nt / -ntmpi / -ntomp, but nothing changes.
> >
> > The mdp file and some statistics are following:
> >
> >  START MDP 
> >
> > title = G6PD wt molecular dynamics (2bhl.pdb) - NPT MD
> >
> > ; Run parameters
> > integrator  = md; Algorithm options
> > nsteps  = 2500  ; maximum number of steps to
> > perform [50 ns]
> > dt  = 0.002 ; 2 fs = 0.002 ps
> >
> > ; Output control
> > nstxout= 1 ; [steps] freq to write coordinates to
> > trajectory, the last coordinates are always written
> > nstvout= 1 ; [steps] freq to write velocities to
> > trajectory, the last velocities are always written
> > nstlog  = 1 ; [steps] freq to write energies to log
> > file, the last energies are always written
> > nstenergy = 1  ; [steps] write energies to disk
> > every nstenergy steps
> > nstxtcout  = 1 ; [steps] freq to write coordinates to
> > xtc trajectory
> > xtc_precision   = 1000  ; precision to write to xtc trajectory
> > (1000 = default)
> > xtc_grps= system; which coordinate
> > group(s) to write to disk
> > energygrps  = system; or System / which energy
> > group(s) to writk
> >
> > ; Bond parameters
> > continuation= yes   ; restarting from npt
> > constraints = all-bonds ; Bond types to replace by constraints
> > constraint_algorithm= lincs ; holonomic constraints
> > lincs_iter  = 1 ; accuracy of LINCS
> > lincs_order = 4 ; also related to
> > accuracy
> > lincs_warnangle  = 30; [degrees] maximum angle that a bond can
> > rotate before LINCS will complain
> >
> 
> That seems a little loose for constraints but setting that up and 
> checking it's conserving energy and preserving bond lengths is something 
> you'll have to do yourself
> 
> Richard
> > ; Neighborsearching
> > ns_type = grid  ; method of updating neighbor list
> > cutoff-scheme = Verlet
> > nstlist = 10; [steps] frequence to update
> > neighbor list (10)
> > rlist = 1.0   ; [nm] cut-off distance for the
> > short-range neighbor list  (1 default)
> > rcoulomb  = 1.0   ; [nm] long range electrostatic cut-off
> > rvdw  = 1.0   ; [nm]  long range Van der Waals cut-off
> >
> > ; Electrostatics
> > coulombtype= PME  ; treatment of long range electrostatic
> > interactions
> > vdwtype = cut-off   ; treatment of Van der Waals
> > interactions
> >
> > ; Periodic boundary conditions
> > pbc = xyz
> >
> > ; Dispersion correction
> > DispCorr= EnerPres  ; appling long
> > range dispersion

Re: [gmx-users] gpu cluster explanation

[Richard Broadbent]

On 12/07/13 13:26, Francesco wrote:

Hi all,
I'm working with a 200K atoms system (protein + explicit water) and
after a while using a cpu cluster I had to switch to a gpu cluster.
I read both Acceleration and parallelization and Gromacs-gpu
documentation pages
(http://www.gromacs.org/Documentation/Acceleration_and_parallelization
and
http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GROMACS-OpenMM)
but it's a bit confusing and I need help to understand if I really have
understood correctly. :)
I have 2 type of nodes:
3gpu ( NVIDIA Tesla M2090) and 2 cpu 6cores each (Intel Xeon E5649 @
2.53GHz)
8gpu and 2 cpu (6 cores each)

1) I can only have 1 MPI per gpu, meaning that with 3 gpu I can have 3
MPI max.
2) because I have 12 cores I can open 4 OPenMP threads x MPI, because
4x3= 12

now if I have a node with 8 gpu, I can use 4 gpu:
4 MPI and 3 OpenMP
is it right?
is it possible to use 8 gpu and 8 cores only?


you could set -ntomp 0, however and setup mpi/thread mpi to use 8 cores. 
However, a system that unbalanced (huge amount of gpu power to 
comparatively little cpu power) is unlikely to get great performance.


Using gromacs 4.6.2 and 144 cpu cores I reach 35 ns/day, while with 3
gpu  and 12 cores I get 9-11 ns/day.

That slowdown is in line with what I got when I tried a similar cpu-gpu 
setup. That said other's might have some advice that will improve your 
performance.



the command that I use is:
mdrun -dlb yes -s input_50.tpr -deffnm 306s_50 -v
with n° gpu set via script :
#BSUB -n 3

I also tried to set -npme / -nt / -ntmpi / -ntomp, but nothing changes.

The mdp file and some statistics are following:

 START MDP 

title = G6PD wt molecular dynamics (2bhl.pdb) - NPT MD

; Run parameters
integrator  = md; Algorithm options
nsteps  = 2500  ; maximum number of steps to
perform [50 ns]
dt  = 0.002 ; 2 fs = 0.002 ps

; Output control
nstxout= 1 ; [steps] freq to write coordinates to
trajectory, the last coordinates are always written
nstvout= 1 ; [steps] freq to write velocities to
trajectory, the last velocities are always written
nstlog  = 1 ; [steps] freq to write energies to log
file, the last energies are always written
nstenergy = 1  ; [steps] write energies to disk
every nstenergy steps
nstxtcout  = 1 ; [steps] freq to write coordinates to
xtc trajectory
xtc_precision   = 1000  ; precision to write to xtc trajectory
(1000 = default)
xtc_grps= system; which coordinate
group(s) to write to disk
energygrps  = system; or System / which energy
group(s) to writk

; Bond parameters
continuation= yes   ; restarting from npt
constraints = all-bonds ; Bond types to replace by constraints
constraint_algorithm= lincs ; holonomic constraints
lincs_iter  = 1 ; accuracy of LINCS
lincs_order = 4 ; also related to
accuracy
lincs_warnangle  = 30; [degrees] maximum angle that a bond can
rotate before LINCS will complain



That seems a little loose for constraints but setting that up and 
checking it's conserving energy and preserving bond lengths is something 
you'll have to do yourself


Richard

; Neighborsearching
ns_type = grid  ; method of updating neighbor list
cutoff-scheme = Verlet
nstlist = 10; [steps] frequence to update
neighbor list (10)
rlist = 1.0   ; [nm] cut-off distance for the
short-range neighbor list  (1 default)
rcoulomb  = 1.0   ; [nm] long range electrostatic cut-off
rvdw  = 1.0   ; [nm]  long range Van der Waals cut-off

; Electrostatics
coulombtype= PME  ; treatment of long range electrostatic
interactions
vdwtype = cut-off   ; treatment of Van der Waals
interactions

; Periodic boundary conditions
pbc = xyz

; Dispersion correction
DispCorr= EnerPres  ; appling long
range dispersion corrections

; Ewald
fourierspacing= 0.12; grid spacing for FFT  -
controll the higest magnitude of wave vectors (0.12)
pme_order = 4 ; interpolation order for PME, 4 = cubic
ewald_rtol= 1e-5  ; relative strength of Ewald-shifted
potential at rcoulomb

; Temperature coupling
tcoupl  = nose-hoover   ; temperature
coupling with Nose-Hoover ensemble
tc_grps = Protein Non-Protein
tau_t   = 0.40.4; [ps]
time constant
ref_t   = 310310; [K]
reference temperature for coupling [310 = 28°C

; Pressure coupling
pcoupl  = parrinello-rahman
pcoupltype= isotro

[gmx-users] gpu cluster explanation

[Francesco]

Hi all,
I'm working with a 200K atoms system (protein + explicit water) and
after a while using a cpu cluster I had to switch to a gpu cluster.
I read both Acceleration and parallelization and Gromacs-gpu
documentation pages
(http://www.gromacs.org/Documentation/Acceleration_and_parallelization
and
http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GROMACS-OpenMM)
but it's a bit confusing and I need help to understand if I really have
understood correctly. :)
I have 2 type of nodes:
3gpu ( NVIDIA Tesla M2090) and 2 cpu 6cores each (Intel Xeon E5649 @
2.53GHz)
8gpu and 2 cpu (6 cores each)

1) I can only have 1 MPI per gpu, meaning that with 3 gpu I can have 3
MPI max.
2) because I have 12 cores I can open 4 OPenMP threads x MPI, because
4x3= 12

now if I have a node with 8 gpu, I can use 4 gpu:
4 MPI and 3 OpenMP 
is it right?
is it possible to use 8 gpu and 8 cores only?

Using gromacs 4.6.2 and 144 cpu cores I reach 35 ns/day, while with 3
gpu  and 12 cores I get 9-11 ns/day.

the command that I use is:
mdrun -dlb yes -s input_50.tpr -deffnm 306s_50 -v
with n° gpu set via script :
#BSUB -n 3

I also tried to set -npme / -nt / -ntmpi / -ntomp, but nothing changes.

The mdp file and some statistics are following:

 START MDP 

title = G6PD wt molecular dynamics (2bhl.pdb) - NPT MD

; Run parameters
integrator  = md; Algorithm options
nsteps  = 2500  ; maximum number of steps to
perform [50 ns]
dt  = 0.002 ; 2 fs = 0.002 ps

; Output control
nstxout= 1 ; [steps] freq to write coordinates to
trajectory, the last coordinates are always written
nstvout= 1 ; [steps] freq to write velocities to
trajectory, the last velocities are always written
nstlog  = 1 ; [steps] freq to write energies to log
file, the last energies are always written
nstenergy = 1  ; [steps] write energies to disk
every nstenergy steps
nstxtcout  = 1 ; [steps] freq to write coordinates to
xtc trajectory
xtc_precision   = 1000  ; precision to write to xtc trajectory
(1000 = default)
xtc_grps= system; which coordinate
group(s) to write to disk 
energygrps  = system; or System / which energy
group(s) to writk

; Bond parameters
continuation= yes   ; restarting from npt
constraints = all-bonds ; Bond types to replace by constraints
constraint_algorithm= lincs ; holonomic constraints
lincs_iter  = 1 ; accuracy of LINCS
lincs_order = 4 ; also related to
accuracy
lincs_warnangle  = 30; [degrees] maximum angle that a bond can
rotate before LINCS will complain

; Neighborsearching
ns_type = grid  ; method of updating neighbor list
cutoff-scheme = Verlet
nstlist = 10; [steps] frequence to update
neighbor list (10)
rlist = 1.0   ; [nm] cut-off distance for the
short-range neighbor list  (1 default)
rcoulomb  = 1.0   ; [nm] long range electrostatic cut-off
rvdw  = 1.0   ; [nm]  long range Van der Waals cut-off

; Electrostatics
coulombtype= PME  ; treatment of long range electrostatic
interactions  
vdwtype = cut-off   ; treatment of Van der Waals
interactions

; Periodic boundary conditions
pbc = xyz  

; Dispersion correction
DispCorr= EnerPres  ; appling long
range dispersion corrections

; Ewald
fourierspacing= 0.12; grid spacing for FFT  -
controll the higest magnitude of wave vectors (0.12)
pme_order = 4 ; interpolation order for PME, 4 = cubic
ewald_rtol= 1e-5  ; relative strength of Ewald-shifted
potential at rcoulomb

; Temperature coupling
tcoupl  = nose-hoover   ; temperature
coupling with Nose-Hoover ensemble
tc_grps = Protein Non-Protein
tau_t   = 0.40.4; [ps]
time constant
ref_t   = 310310; [K]
reference temperature for coupling [310 = 28°C

; Pressure coupling
pcoupl  = parrinello-rahman 
pcoupltype= isotropic   ;
uniform scaling of box vect
tau_p   = 2.0  
; [ps] time constant
ref_p   = 1.0   
   ; [bar] reference pressure for coupling
compressibility = 4.5e-5   
; [bar^-1] isothermal compressibility of water
refcoord_scaling= com   
   ; have a look at GROMACS documentation 7.

; Velocity generation
gen_vel = no 

[gmx-users] GPU cannot be detected

[Albert]

Hello:

 I've installed Gromacs-4.6.2 in GPU cluster with following configurations:

CC=icc FC=ifort F77=ifort CXX=icpc 
CMAKE_PREFIX_PATH=/export/intel/cmkl/include/fftw:/export/mpi/mvapich2-1.8-rhes6 
cmake .. -DGMX_MPI=ON 
-DCMAKE_INSTALL_PREFIX=/home/albert/install/gromacs -DGMX_GPU=ON 
-DBUILD_SHARED_LIBS=OFF -DCUDA_TOOLKIT_ROOT_DIR=/export/cuda5.0 
-DFFTWF_LIBRARY=/home/albert/install/fftw-3.3.3/lib/libfftw3f.so



However, when I submit job with qsub, Gromacs failed with following 
messages:




No GPUs detected on host c0204

Can not set thread affinities on the current platform. On NUMA systems this
can cause performance degradation. If you think your platform should 
support

setting affinities, contact the GROMACS developers.
--
mpirun noticed that process rank 2 with PID 6 on node c0205 exited 
on signal 4 (Illegal instruction).

--

I am just wondering how can we solve this problem?

thank you very much

Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: Re: Re: [gmx-users] GPU-based workstation

[James Starlight]

Back to my question
I want to build gpu-based workstation based onto 2 titans geforces.

My current budget allow me only hight-end  6nodes core i 7-3930  and MB
with 5 PCI-E (like Asus rampage IV series). Would this system be balanced
with two GPUs ? Should I use two 6-8 nodes XEONS instead of i7?

James

2013/5/29 James Starlight 

> Dear Dr. Pall!
>
> Thank you for your suggestions!
>
> Asumming that I have budget of 5000 $ and I want to build gpu-based
> desktop on this money.
>
> Previously I've used single 4 core i5 with GTX 670 and obtain average 10
> ns\day performance for the 70k atoms systems (1.0 cutoffs, no virtual sites
> , sd integrator).
>
> Now I'd like to build system based on 2 hight-end GeForces (e.g like
> TITAN).
> Should that system include 2 cpu's for good balancing? (e.g two 6 nodes
> XEONS with faster clocks for instance could be better for simulations than
> i7, couldnt it?)
>
> What addition properties to the MB should I consider for such system ?
>
> James
>
>
> 2013/5/28 lloyd riggs 
>
>> Dear Dr. Pali,
>>
>> Thank you,
>>
>> Stephan Watkins
>>
>> *Gesendet:* Dienstag, 28. Mai 2013 um 19:50 Uhr
>> *Von:* "Szilárd Páll" 
>>
>> *An:* "Discussion list for GROMACS users" 
>> *Betreff:* Re: Re: [gmx-users] GPU-based workstation
>> Dear all,
>>
>> As far as I understand, the OP is interested in hardware for *running*
>> GROMACS 4.6 rather than developing code. or running LINPACK.
>>
>>
>> To get best performance it is important to use a machine with hardware
>> balanced for GROMACS' workloads. Too little GPU resources will result
>> in CPU idling; too much GPU resources will lead to the runs being CPU
>> or multi-GPU scaling bound and above a certain level GROMACS won't be
>> able to make use of additional GPUs.
>>
>> Of course, the balance will depend both on hardware and simulation
>> settings (mostly the LJ cut-off used).
>>
>> An additional factor to consider is typical system size. To reach near
>> peak pair-force throughput on GPUs you typically need >20k-40k
>> particles/GPU (depends on the architecture) and throughput drops below
>> these values. Hence, in most cases it is preferred to use fewer and
>> faster GPUs rather than more.
>>
>> Without knowing the budgdet and indented use of the machine it is hard
>> to make suggestions, but I would say for a budget desktop box a
>> quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
>> fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
>> If you're considering dual-socket workstations, I suggest you go with
>> the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
>> GHz), otherwise you may not see as much benefit as you would expect
>> based on the insane price tag (especially if you compare to an i7
>> 3939K or its IVB successor).
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>> On Sat, May 25, 2013 at 1:02 PM, lloyd riggs  wrote:
>> > More RAM the better, and the best I have seen is 4 GPU work station. I
>> can
>> > use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
>> > really 3-4 GPU, except the tyan mentioned (there designed as blades so
>> an 8
>> > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
>> > though with GPU's, as on a board there packed, so extra cooling things
>> may
>> > help not blow a GPU, but I would look for good ones (ask around), as
>> its a
>> > video game market and they go for looks even though its in casing? The
>> > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I
>> dont
>> > know performance wise, the onboard GPU, the more RAM the marrier...so
>> yes,
>> > normal work stations you can get 4 GPU's for a 300 US$ board, but then
>> the
>> > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered
>> abroad is
>> > also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
>> > software, not Gromacs, so would be nice to see performance...for a
>> small 100
>> > atom molecule and 500 solvent, using just the CPU I get it to run 5-10
>> > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
>> solvent
>> > eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps
>> > eq's
>> >
>> > Stephan
>> >
>> > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
>> > Von: &qu

Re: [gmx-users] GPU / CPU load imblance

[Justin Lemkul]

On 6/25/13 6:33 PM, Dwey wrote:

Hi gmx-users,

 I used  8-cores AMD CPU  with a GTX680 GPU [ with 1536 CUDA Cores]  to
run an example of Umbrella Sampling provided by Justin.
I am happy that GPU acceleration indeed helps me reduce significant time (
from 34 hours to 7 hours)  of computation in this example.
However, I found there was a NOTE on the screen like

++
  The GPU has >20% more load than the CPU. This imbalance causes
performance loss, consider using a shorter cut-off and a finer PME grid
  ++

Given a 20% load imbalance, I wonder if someone can give suggestions as to
how to avoid performance loss in terms of hardware (GPU/CPU)
improvement  or  the modification of  mdp file (see below).



I would avoid tweaking the .mdp settings.  There have been several reports where 
people hacked at nonbonded cutoffs to get better performance, and it resulted in 
totally useless output.  These settings are part of the force field.  Avoid 
changing them.



In terms of hardware,  dose this NOTE suggest that I should use a
higher-capacity GPU like GTX 780 [ with 2304 CUDA Cores] to balance load or
catch up speed  ?
If so,   can it help by adding  another card with  GTX 680 GPU in the same
box ?  but will it cause GPU/CPU imbalance load  again, which two GPU keep
waiting for 8-cores CPU  ?


There has been a lot of discussion on hardware, GPU/CPU balancing, etc. in 
recent days.  Please check the archive.  Some of the threads are quite detailed.


-Justin



Second,

++
Force evaluation time GPU/CPU: 4.006 ms/2.578 ms = 1.554
For optimal performance this ratio should be close to 1
++

I have no idea how this is evaluated by 4.006 ms and 2.578 ms for GPU and
CPU time, respectively.

It will be very helpful to modify  the attached mdp for a better
load balance between GPU and CPU.

I appreciate kind advice and hints to improve this mdp file.

Thanks,

Dwey

### courtesy  to  Justin #

title   = Umbrella pulling simulation
define  = -DPOSRES_B
; Run parameters
integrator  = md
dt  = 0.002
tinit   = 0
nsteps  = 500   ; 10 ns
nstcomm = 10
; Output parameters
nstxout = 5 ; every 100 ps
nstvout = 5
nstfout = 5000
nstxtcout   = 5000  ; every 10 ps
nstenergy   = 5000
; Bond parameters
constraint_algorithm= lincs
constraints = all-bonds
continuation= yes
; Single-range cutoff scheme
nstlist = 5
ns_type = grid
rlist   = 1.4
rcoulomb= 1.4
rvdw= 1.4
; PME electrostatics parameters
coulombtype = PME
fourierspacing  = 0.12
fourier_nx  = 0
fourier_ny  = 0
fourier_nz  = 0
pme_order   = 4
ewald_rtol  = 1e-5
optimize_fft= yes
; Berendsen temperature coupling is on in two groups
Tcoupl  = Nose-Hoover
tc_grps = Protein   Non-Protein
tau_t   = 0.5   0.5
ref_t   = 310   310
; Pressure coupling is on
Pcoupl  = Parrinello-Rahman
pcoupltype  = isotropic
tau_p   = 1.0
compressibility = 4.5e-5
ref_p   = 1.0
refcoord_scaling = com
; Generate velocities is off
gen_vel = no
; Periodic boundary conditions are on in all directions
pbc = xyz
; Long-range dispersion correction
DispCorr= EnerPres
cutoff-scheme   = Verlet
; Pull code
pull= umbrella
pull_geometry   = distance
pull_dim= N N Y
pull_start  = yes
pull_ngroups= 1
pull_group0 = Chain_B
pull_group1 = Chain_A
pull_init1  = 0
pull_rate1  = 0.0
pull_k1 = 1000  ; kJ mol^-1 nm^-2
pull_nstxout= 1000  ; every 2 ps
pull_nstfout= 1000  ; every 2 ps



--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU / CPU load imblance

[Dwey]

Hi gmx-users,

I used  8-cores AMD CPU  with a GTX680 GPU [ with 1536 CUDA Cores]  to
run an example of Umbrella Sampling provided by Justin.
I am happy that GPU acceleration indeed helps me reduce significant time (
from 34 hours to 7 hours)  of computation in this example.
However, I found there was a NOTE on the screen like

++
 The GPU has >20% more load than the CPU. This imbalance causes
performance loss, consider using a shorter cut-off and a finer PME grid
 ++

Given a 20% load imbalance, I wonder if someone can give suggestions as to
how to avoid performance loss in terms of hardware (GPU/CPU)
improvement  or  the modification of  mdp file (see below).

In terms of hardware,  dose this NOTE suggest that I should use a
higher-capacity GPU like GTX 780 [ with 2304 CUDA Cores] to balance load or
catch up speed  ?
If so,   can it help by adding  another card with  GTX 680 GPU in the same
box ?  but will it cause GPU/CPU imbalance load  again, which two GPU keep
waiting for 8-cores CPU  ?

Second,

++
Force evaluation time GPU/CPU: 4.006 ms/2.578 ms = 1.554
For optimal performance this ratio should be close to 1
++

I have no idea how this is evaluated by 4.006 ms and 2.578 ms for GPU and
CPU time, respectively.

It will be very helpful to modify  the attached mdp for a better
load balance between GPU and CPU.

I appreciate kind advice and hints to improve this mdp file.

Thanks,

Dwey

### courtesy  to  Justin #

title   = Umbrella pulling simulation
define  = -DPOSRES_B
; Run parameters
integrator  = md
dt  = 0.002
tinit   = 0
nsteps  = 500   ; 10 ns
nstcomm = 10
; Output parameters
nstxout = 5 ; every 100 ps
nstvout = 5
nstfout = 5000
nstxtcout   = 5000  ; every 10 ps
nstenergy   = 5000
; Bond parameters
constraint_algorithm= lincs
constraints = all-bonds
continuation= yes
; Single-range cutoff scheme
nstlist = 5
ns_type = grid
rlist   = 1.4
rcoulomb= 1.4
rvdw= 1.4
; PME electrostatics parameters
coulombtype = PME
fourierspacing  = 0.12
fourier_nx  = 0
fourier_ny  = 0
fourier_nz  = 0
pme_order   = 4
ewald_rtol  = 1e-5
optimize_fft= yes
; Berendsen temperature coupling is on in two groups
Tcoupl  = Nose-Hoover
tc_grps = Protein   Non-Protein
tau_t   = 0.5   0.5
ref_t   = 310   310
; Pressure coupling is on
Pcoupl  = Parrinello-Rahman
pcoupltype  = isotropic
tau_p   = 1.0
compressibility = 4.5e-5
ref_p   = 1.0
refcoord_scaling = com
; Generate velocities is off
gen_vel = no
; Periodic boundary conditions are on in all directions
pbc = xyz
; Long-range dispersion correction
DispCorr= EnerPres
cutoff-scheme   = Verlet
; Pull code
pull= umbrella
pull_geometry   = distance
pull_dim= N N Y
pull_start  = yes
pull_ngroups= 1
pull_group0 = Chain_B
pull_group1 = Chain_A
pull_init1  = 0
pull_rate1  = 0.0
pull_k1 = 1000  ; kJ mol^-1 nm^-2
pull_nstxout= 1000  ; every 2 ps
pull_nstfout= 1000  ; every 2 ps
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU ECC question

[Szilárd Páll]

On Sat, Jun 8, 2013 at 9:21 PM, Albert  wrote:
> Hello:
>
>  Recently I found a strange question about Gromacs-4.6.2 on GPU workstaion.
> In my GTX690 machine, when I run md production I found that the ECC is on.
> However, in my another GTX590 machine, I found the ECC was off:
>
> 4 GPUs detected:
>   #0: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #1: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #2: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #3: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>
> moreover, there is only two GTX590 in the machine, I don't know why Gromacs
> claimed 4 GPU detected. However, in my another Linux machine which also have
> two GTX590, Gromacs-4.6.2 only find 2 GPU, and ECC is still off.
>
> I am just wondering:
>
> (1) why in GTX690 the ECC can be on while it is off in my GTX590? I compiled
> Gromacs with the same options and the same version of intel compiler

Unless your 690 is in fact a Tesla K10 it does surely not support ECC!
Note that ECC is not something I personally think you really need.

>
> (2) why in machines both of physically installed two GTX590 cards, one of
> them was detected with 4 GPU while the other was claimed contains two GPU?
>

Both GTX 590 and 690 are dual-chip boards which means two independent
processing units with their own memory mounted on the same card and
connected by a PCI switch (NVIDIA NF200). Hence, the two GPUs on these
dual-chip boards will be enumerated as a separate devices. You can
double-check this in nvidia-smi which should give the same devices as
what mdrun reports. I suspect that one of the GPUs which is shown to
have only two GPUs suffers from some hardware or software issues.


Regards,
Szilard

> thank you very much
>
> best
> Albert
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU ECC question

[Albert]

Hello:

 Recently I found a strange question about Gromacs-4.6.2 on GPU 
workstaion. In my GTX690 machine, when I run md production I found that 
the ECC is on. However, in my another GTX590 machine, I found the ECC 
was off:


4 GPUs detected:
  #0: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #3: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible

moreover, there is only two GTX590 in the machine, I don't know why 
Gromacs claimed 4 GPU detected. However, in my another Linux machine 
which also have two GTX590, Gromacs-4.6.2 only find 2 GPU, and ECC is 
still off.


I am just wondering:

(1) why in GTX690 the ECC can be on while it is off in my GTX590? I 
compiled Gromacs with the same options and the same version of intel 
compiler


(2) why in machines both of physically installed two GTX590 cards, one 
of them was detected with 4 GPU while the other was claimed contains two 
GPU?


thank you very much

best
Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Aw: Re: [gmx-users] GPU problem

[lloyd riggs]

 

Thanks, thats exact what I was looking for.

 

Stephan


Gesendet: Dienstag, 04. Juni 2013 um 22:28 Uhr
Von: "Justin Lemkul" 
An: "Discussion list for GROMACS users" 
Betreff: Re: [gmx-users] GPU problem



On 6/4/13 3:52 PM, lloyd riggs wrote:
> Dear All or anyone,
> A stupid question. Is there an script anyone knows of to convert a 53a6ff from
> .top redirects to the gromacs/top directory to something like a ligand .itp?
> This is usefull at the moment. Example:
> [bond]
> 6 7 2 gb_5
> to
> [bonds]
> ; ai aj fu c0, c1, ...
> 6 7 2 0.139 1080.0 0.139 1080.0 ; C CH
> for everything (a protein/DNA complex) inclusive of angles, dihedrials?
> Ive been playing with some of the gromacs user supplied files, but nothing yet.

Sounds like something grompp -pp should take care of.

-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing list gmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists



-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU problem

[Justin Lemkul]

On 6/4/13 3:52 PM, lloyd riggs wrote:

Dear All or anyone,
A stupid question.  Is there an script anyone knows of to convert a 53a6ff from
.top redirects to the gromacs/top directory to something like a ligand .itp?
This is usefull at the moment.  Example:
[bond]
 6 7 2gb_5
to
[bonds]
; ai  aj  fuc0, c1, ...
   6  7   20.139  1080.00.139  1080.0 ;   C  CH
for everything (a protein/DNA complex) inclusive of angles, dihedrials?
Ive been playing with some of the gromacs user supplied files, but nothing yet.


Sounds like something grompp -pp should take care of.

-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

RE:[gmx-users] GPU problem

[lloyd riggs]

 

Dear All or anyone,

 

A stupid question.  Is there an script anyone knows of to convert a 53a6ff from .top redirects to the gromacs/top directory to something like a ligand .itp?  This is usefull at the moment.  Example:

 

[bond]

    6 7 2    gb_5

 

to

 

[bonds]

; ai  aj  fu    c0, c1, ...

  6  7   2    0.139  1080.0    0.139  1080.0 ;   C  CH  

 

for everything (a protein/DNA complex) inclusive of angles, dihedrials?

 

Ive been playing with some of the gromacs user supplied files, but nothing yet.

 

Stephan Watkins
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU problem

[Szilárd Páll]

"-nt" is mostly a backward compatibility option and sets the total
number of threads (per rank). Instead, you should set both "-ntmpi"
(or -np with MPI) and "-ntomp". However, note that unless a single
mdrun uses *all* cores/hardware threads on a node, it won't pin the
threads to cores. Failing to pin threads will lead to considerable
performance degradation; just tried and depending on how (un)lucky the
thread placement and migration is, I get 1.5-2x performance
degradation with running two mdrun-s on a single dual-socket node
without pining threads.

My advise is (yet again) that you should check the
http://www.gromacs.org/Documentation/Acceleration_and_parallelization
wiki page, in particular the section on how to run simulations. If
things are not, clear please ask for clarification - input and
constructive criticism should help us improve the wiki.

We have been patiently pointing everyone to the wiki, so asking
without reading up first is neither productive nor really fair.

Cheers,
--
Szilárd


On Tue, Jun 4, 2013 at 11:22 AM, Chandan Choudhury  wrote:
> Hi Albert,
>
> I think using -nt flag (-nt=16) with mdrun would solve your problem.
>
> Chandan
>
>
> --
> Chandan kumar Choudhury
> NCL, Pune
> INDIA
>
>
> On Tue, Jun 4, 2013 at 12:56 PM, Albert  wrote:
>
>> Dear:
>>
>>  I've got four GPU in one workstation. I am trying to run two GPU job with
>> command:
>>
>> mdrun -s md.tpr -gpu_id 01
>> mdrun -s md.tpr -gpu_id 23
>>
>> there are 32 CPU in this workstation. I found that each job trying to use
>> the whole CPU, and there are 64 sub job when these two GPU mdrun submitted.
>>  Moreover, one of the job stopped after short of running, probably because
>> of the CPU issue.
>>
>> I am just wondering, how can we distribute CPU when we run two GPU job in
>> a single workstation?
>>
>> thank you very much
>>
>> best
>> Albert
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users
>> * Please search the archive at http://www.gromacs.org/**
>> Support/Mailing_Lists/Searchbefore
>>  posting!
>> * Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists
>>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU problem

[Albert]

On 06/04/2013 11:22 AM, Chandan Choudhury wrote:

Hi Albert,

I think using -nt flag (-nt=16) with mdrun would solve your problem.

Chandan



thank you so much.

it works well now.

ALBERT
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU problem

[Chandan Choudhury]

Hi Albert,

I think using -nt flag (-nt=16) with mdrun would solve your problem.

Chandan


--
Chandan kumar Choudhury
NCL, Pune
INDIA


On Tue, Jun 4, 2013 at 12:56 PM, Albert  wrote:

> Dear:
>
>  I've got four GPU in one workstation. I am trying to run two GPU job with
> command:
>
> mdrun -s md.tpr -gpu_id 01
> mdrun -s md.tpr -gpu_id 23
>
> there are 32 CPU in this workstation. I found that each job trying to use
> the whole CPU, and there are 64 sub job when these two GPU mdrun submitted.
>  Moreover, one of the job stopped after short of running, probably because
> of the CPU issue.
>
> I am just wondering, how can we distribute CPU when we run two GPU job in
> a single workstation?
>
> thank you very much
>
> best
> Albert
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU problem

[Albert]

Dear:

 I've got four GPU in one workstation. I am trying to run two GPU job 
with command:


mdrun -s md.tpr -gpu_id 01
mdrun -s md.tpr -gpu_id 23

there are 32 CPU in this workstation. I found that each job trying to 
use the whole CPU, and there are 64 sub job when these two GPU mdrun 
submitted.  Moreover, one of the job stopped after short of running, 
probably because of the CPU issue.


I am just wondering, how can we distribute CPU when we run two GPU job 
in a single workstation?


thank you very much

best
Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: Re: Re: [gmx-users] GPU-based workstation

[James Starlight]

Dear Dr. Pall!

Thank you for your suggestions!

Asumming that I have budget of 5000 $ and I want to build gpu-based desktop
on this money.

Previously I've used single 4 core i5 with GTX 670 and obtain average 10
ns\day performance for the 70k atoms systems (1.0 cutoffs, no virtual sites
, sd integrator).

Now I'd like to build system based on 2 hight-end GeForces (e.g like TITAN).
Should that system include 2 cpu's for good balancing? (e.g two 6 nodes
XEONS with faster clocks for instance could be better for simulations than
i7, couldnt it?)

What addition properties to the MB should I consider for such system ?

James

2013/5/28 lloyd riggs 

> Dear Dr. Pali,
>
> Thank you,
>
> Stephan Watkins
>
> *Gesendet:* Dienstag, 28. Mai 2013 um 19:50 Uhr
> *Von:* "Szilárd Páll" 
>
> *An:* "Discussion list for GROMACS users" 
> *Betreff:* Re: Re: [gmx-users] GPU-based workstation
> Dear all,
>
> As far as I understand, the OP is interested in hardware for *running*
> GROMACS 4.6 rather than developing code. or running LINPACK.
>
>
> To get best performance it is important to use a machine with hardware
> balanced for GROMACS' workloads. Too little GPU resources will result
> in CPU idling; too much GPU resources will lead to the runs being CPU
> or multi-GPU scaling bound and above a certain level GROMACS won't be
> able to make use of additional GPUs.
>
> Of course, the balance will depend both on hardware and simulation
> settings (mostly the LJ cut-off used).
>
> An additional factor to consider is typical system size. To reach near
> peak pair-force throughput on GPUs you typically need >20k-40k
> particles/GPU (depends on the architecture) and throughput drops below
> these values. Hence, in most cases it is preferred to use fewer and
> faster GPUs rather than more.
>
> Without knowing the budgdet and indented use of the machine it is hard
> to make suggestions, but I would say for a budget desktop box a
> quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
> fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
> If you're considering dual-socket workstations, I suggest you go with
> the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
> GHz), otherwise you may not see as much benefit as you would expect
> based on the insane price tag (especially if you compare to an i7
> 3939K or its IVB successor).
>
> Cheers,
> --
> Szilárd
>
>
> On Sat, May 25, 2013 at 1:02 PM, lloyd riggs  wrote:
> > More RAM the better, and the best I have seen is 4 GPU work station. I
> can
> > use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
> > really 3-4 GPU, except the tyan mentioned (there designed as blades so
> an 8
> > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
> > though with GPU's, as on a board there packed, so extra cooling things
> may
> > help not blow a GPU, but I would look for good ones (ask around), as its
> a
> > video game market and they go for looks even though its in casing? The
> > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I
> dont
> > know performance wise, the onboard GPU, the more RAM the marrier...so
> yes,
> > normal work stations you can get 4 GPU's for a 300 US$ board, but then
> the
> > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad
> is
> > also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> > software, not Gromacs, so would be nice to see performance...for a small
> 100
> > atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
> solvent
> > eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps
> > eq's
> >
> > Stephan
> >
> > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> > Von: "James Starlight" 
> > An: "Discussion list for GROMACS users" 
> > Betreff: Re: [gmx-users] GPU-based workstation
> > Dear Dr. Watkins!
> >
> > Thank you for the suggestions!
> >
> > In the local shops I've found only Core i7 with 6 cores (like Core
> > i7-39xx) and 4 cores. Should I obtain much better performance with 6
> cores
> > than with 4 cores in case of i7 cpu (assuming that I run simulation in
> > cpu+gpu mode )?
> >
> > Also you've mentioned about 4 PCeI MD. Does it means that modern
> > work-station could have 4 GPU's in one home-like desktop ? According to
> my
> > current task I suppose that 2 GPU's would be suitable 

Aw: Re: Re: [gmx-users] GPU-based workstation

[lloyd riggs]

Dear Dr. Pali,

 

Thank you,

 

Stephan Watkins

 

Gesendet: Dienstag, 28. Mai 2013 um 19:50 Uhr
Von: "Szilárd Páll" 
An: "Discussion list for GROMACS users" 
Betreff: Re: Re: [gmx-users] GPU-based workstation

Dear all,

As far as I understand, the OP is interested in hardware for *running*
GROMACS 4.6 rather than developing code. or running LINPACK.


To get best performance it is important to use a machine with hardware
balanced for GROMACS' workloads. Too little GPU resources will result
in CPU idling; too much GPU resources will lead to the runs being CPU
or multi-GPU scaling bound and above a certain level GROMACS won't be
able to make use of additional GPUs.

Of course, the balance will depend both on hardware and simulation
settings (mostly the LJ cut-off used).

An additional factor to consider is typical system size. To reach near
peak pair-force throughput on GPUs you typically need >20k-40k
particles/GPU (depends on the architecture) and throughput drops below
these values. Hence, in most cases it is preferred to use fewer and
faster GPUs rather than more.

Without knowing the budgdet and indented use of the machine it is hard
to make suggestions, but I would say for a budget desktop box a
quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
If you're considering dual-socket workstations, I suggest you go with
the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
GHz), otherwise you may not see as much benefit as you would expect
based on the insane price tag (especially if you compare to an i7
3939K or its IVB successor).

Cheers,
--
Szilárd


On Sat, May 25, 2013 at 1:02 PM, lloyd riggs  wrote:
> More RAM the better, and the best I have seen is 4 GPU work station. I can
> use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
> really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8
> or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
> though with GPU's, as on a board there packed, so extra cooling things may
> help not blow a GPU, but I would look for good ones (ask around), as its a
> video game market and they go for looks even though its in casing? The
> external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont
> know performance wise, the onboard GPU, the more RAM the marrier...so yes,
> normal work stations you can get 4 GPU's for a 300 US$ board, but then the
> price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad is
> also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> software, not Gromacs, so would be nice to see performance...for a small 100
> atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> minutes real for 1 ns sim, but tried simple large 800 amino, 25,000 solvent
> eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps
> eq's
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" 
> An: "Discussion list for GROMACS users" 
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs 
>
>> There's also these, but 1 chip runs 6K US, they can get performance up to
>> 2.3 teraflops per chip though double percission...but have no clue about
>> integration with GPU's...Intell also sells their chips on PCIe cards...but
>> get only about 350 Gflops, and run 1K US$.
>>
>> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
>> http://www.xilinx.com/
>>
>> They can design them though to fit a PCIe slot and run about the same, but
>> still need the board, ram etc...
>>
>> Mostly just to dream about, they say you can order them with radiation
>> shielding as well...so...
>>
>> Stephan Watkins
>>
>> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
>> *Von:* "J

Re: Aw: Re: [gmx-users] GPU-based workstation

[Szilárd Páll]

On Sat, May 25, 2013 at 2:16 PM, Broadbent, Richard
 wrote:
> I've been running on my Universities GPU nodes these are one E5-xeon (6-cores 
> 12 threads)  and have 4 Nvidia 690gtx's. My system is 93 000 atoms of DMF 
> under NVE.  The performance has been a little disappointing

That sounds like a very imbalanced system for GROMACS, you have
essentially 8 GPUs with rather poor PCI-E performance (a board share a
single PCI-E bus) and only 12 CPU cores to "drive" the simulation.

~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I
get 5.4ns/day for the same system. On our HPC system using 32 nodes
each with 2 quad-core xeon processors I get 30-40ns/day.

That sounds somewhat low if these are all moderately fast CPUs and GPUs.

> I think that to achieve reasonable performance the system has to be balanced 
> between CPU's and GPU's probably getting 2 high end GPU's and a top end xeon 
> E5 or core i7 would be a good choice.

Indeed. Even two GPUs may be too much - unless the CPU in question is
a very high end i7 or E5.

Cheers,
--
Szilárd

>
>
> Richard
>
> From: lloyd riggs mailto:lloyd.ri...@gmx.ch>>
> Reply-To: Discussion users 
> mailto:gmx-users@gromacs.org>>
> Date: Saturday, 25 May 2013 12:02
> To: Discussion users mailto:gmx-users@gromacs.org>>
> Subject: Aw: Re: [gmx-users] GPU-based workstation
>
> More RAM the better, and the best I have seen is 4 GPU work station.  I can 
> use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is really 
> 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 or 10 
> slot board really holds 8 or 10 GPU's).  There's cooling problems though with 
> GPU's, as on a board there packed, so extra cooling things may help not blow 
> a GPU, but I would look for good ones (ask around), as its a video game 
> market and they go for looks even though its in casing?  The external RAM 
> (not onboard GPU RAM) helps if you do a larger sim, but I dont know 
> performance wise, the onboard GPU, the more RAM the marrier...so yes, normal 
> work stations you can get 4 GPU's for a 300 US$ board, but then the price 
> goes way up (3-4000 US$ for an 8-10 gpu board).  RAM ordered abroad is also 
> cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests software, 
> not Gromacs, so would be nice to see performance...for a small 100 atom 
> molecule and 500 solvent, using just the CPU I get it to run 5-10 minutes 
> real  for 1 ns sim, but tried simple large 800 amino, 25,000 solvent eq (NVT 
> or NPT) runs and they clock at around 1 hour real for say 50 ps eq's
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" mailto:jmsstarli...@gmail.com>>
> An: "Discussion list for GROMACS users" 
> mailto:gmx-users@gromacs.org>>
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs mailto:lloyd.ri...@gmx.ch>>
>
>> There's also these, but 1 chip runs 6K US, they can get performance up to
>> 2.3 teraflops per chip though double percission...but have no clue about
>> integration with GPU's...Intell also sells their chips on PCIe cards...but
>> get only about 350 Gflops, and run 1K US$.
>>
>> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
>> http://www.xilinx.com/
>>
>> They can design them though to fit a PCIe slot and run about the same, but
>> still need the board, ram etc...
>>
>> Mostly just to dream about, they say you can order them with radiation
>> shielding as well...so...
>>
>> Stephan Watkins
>>
>> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
>> *Von:* "James Starlight" 
>> mailto:jmsstarli...@gmail.com>>
>> *An:* "Discussion list for GROMACS users" 
>> mailto:gmx-users@gromacs.org>>
>> *Betreff:* [gmx

Re: Re: [gmx-users] GPU-based workstation

[Szilárd Páll]

Dear all,

As far as I understand, the OP is interested in hardware for *running*
GROMACS 4.6 rather than developing code. or running LINPACK.


To get best performance it is important to use a machine with hardware
balanced for GROMACS' workloads. Too little GPU resources will result
in CPU idling; too much GPU resources will lead to the runs being CPU
or multi-GPU scaling bound and above a certain level GROMACS won't be
able to make use of additional GPUs.

Of course, the balance will depend both on hardware and simulation
settings (mostly the LJ cut-off used).

An additional factor to consider is typical system size. To reach near
peak pair-force throughput on GPUs you typically need >20k-40k
particles/GPU (depends on the architecture) and throughput drops below
these values. Hence, in most cases it is preferred to use fewer and
faster GPUs rather than more.

Without knowing the budgdet and indented use of the machine it is hard
to make suggestions, but I would say for a budget desktop box a
quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a
fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well.
If you're considering dual-socket workstations, I suggest you go with
the higher core-count and higher frequency Intel CPUs (6+ cores >2.2
GHz), otherwise you may not see as much benefit as you would expect
based on the insane price tag (especially if you compare to an i7
3939K or its IVB successor).

Cheers,
--
Szilárd


On Sat, May 25, 2013 at 1:02 PM, lloyd riggs  wrote:
> More RAM the better, and the best I have seen is 4 GPU work station.  I can
> use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is
> really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8
> or 10 slot board really holds 8 or 10 GPU's).  There's cooling problems
> though with GPU's, as on a board there packed, so extra cooling things may
> help not blow a GPU, but I would look for good ones (ask around), as its a
> video game market and they go for looks even though its in casing?  The
> external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont
> know performance wise, the onboard GPU, the more RAM the marrier...so yes,
> normal work stations you can get 4 GPU's for a 300 US$ board, but then the
> price goes way up (3-4000 US$ for an 8-10 gpu board).  RAM ordered abroad is
> also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> software, not Gromacs, so would be nice to see performance...for a small 100
> atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> minutes real  for 1 ns sim, but tried simple large 800 amino, 25,000 solvent
> eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps
> eq's
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" 
> An: "Discussion list for GROMACS users" 
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs 
>
>> There's also these, but 1 chip runs 6K US, they can get performance up to
>> 2.3 teraflops per chip though double percission...but have no clue about
>> integration with GPU's...Intell also sells their chips on PCIe cards...but
>> get only about 350 Gflops, and run 1K US$.
>>
>> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
>> http://www.xilinx.com/
>>
>> They can design them though to fit a PCIe slot and run about the same, but
>> still need the board, ram etc...
>>
>> Mostly just to dream about, they say you can order them with radiation
>> shielding as well...so...
>>
>> Stephan Watkins
>>
>> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
>> *Von:* "James Starlight" 
>> *An:* "Discussion list for GROMACS users" 
>> *Betreff:* [gmx-users] GPU-based workstation
>> Dear Gromacs Users!
>>
>>
>> I'd like to build new workstation 

Re: Re: Re: [gmx-users] GPU-based workstation

[James Starlight]

On Nvidia benchmarks I've found suggestions of using of the two 6 cores CPU
for systems with the 2 GPU.

Assuming that I'll be using two 680 GTX cards with 256 bits and 4gb ram
(not a profesional nvidia cards like TESLA)
what CPU's could give me the best performance- 1 i7 of 8 cores
or 2 Xeons e5 with 6 cores ? Does it meaningful to use 2 separate CPU's
with several nodes each for the 2 GPU's ?

James

2013/5/26 lloyd riggs 

>
> You can also look at profilling on varied web sites, the high end Nvidia
> run only slightly better than the 2 year old ones, from an individual point
> not worth the money yet, but if you have the money? as I've been browsing.
>
> Also, the sim I did on the cluster was 180-190,000 atoms so the exact same
> performance the other person had.
>
> Stephan
>  *Gesendet:* Samstag, 25. Mai 2013 um 15:19 Uhr
> *Von:* "James Starlight" 
>
> *An:* "Discussion list for GROMACS users" 
> *Betreff:* Re: Aw: Re: [gmx-users] GPU-based workstation
> Richard,
>
> thanks for suggestion!
>
> Assuming that I'm using 2 high end GeForce's what performance be better
>
> 1) in case of one i7 (4 or 6 nodes ) ?
>
> 2) in case of 8 core Xeon like CPU Intel Xeon E5-2650 2.0 GHz / 8core
>
> What properties of MB should take into account primarily for such
> Xenon-based system. Does such MBs support multi-GPU ( I noticed that many
> such MBs lack for PCI)?
>
> James
>
> 2013/5/25 Broadbent, Richard 
>
> > I've been running on my Universities GPU nodes these are one E5-xeon
> > (6-cores 12 threads) and have 4 Nvidia 690gtx's. My system is 93 000
> atoms
> > of DMF under NVE. The performance has been a little disappointing
> > ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I
> get
> > 5.4ns/day for the same system. On our HPC system using 32 nodes each
> with 2
> > quad-core xeon processors I get 30-40ns/day.
> >
> > I think that to achieve reasonable performance the system has to be
> > balanced between CPU's and GPU's probably getting 2 high end GPU's and a
> > top end xeon E5 or core i7 would be a good choice.
> >
> >
> > Richard
> >
> > From: lloyd riggs mailto:lloyd.ri...@gmx.ch>>
> > Reply-To: Discussion users  > gmx-users@gromacs.org>>
> > Date: Saturday, 25 May 2013 12:02
> > To: Discussion users mailto:gmx-users@gromacs.org
> >>
> > Subject: Aw: Re: [gmx-users] GPU-based workstation
> >
> > More RAM the better, and the best I have seen is 4 GPU work station. I
> > can use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
> > really 3-4 GPU, except the tyan mentioned (there designed as blades so
> an 8
> > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
> > though with GPU's, as on a board there packed, so extra cooling things
> may
> > help not blow a GPU, but I would look for good ones (ask around), as its
> a
> > video game market and they go for looks even though its in casing? The
> > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I
> dont
> > know performance wise, the onboard GPU, the more RAM the marrier...so
> yes,
> > normal work stations you can get 4 GPU's for a 300 US$ board, but then
> the
> > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad
> > is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on
> tests
> > software, not Gromacs, so would be nice to see performance...for a small
> > 100 atom molecule and 500 solvent, using just the CPU I get it to run
> 5-10
> > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
> > solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say
> > 50 ps eq's
> >
> > Stephan
> >
> > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> > Von: "James Starlight"  > jmsstarli...@gmail.com>>
> > An: "Discussion list for GROMACS users"  > gmx-users@gromacs.org>>
> > Betreff: Re: [gmx-users] GPU-based workstation
> > Dear Dr. Watkins!
> >
> > Thank you for the suggestions!
> >
> > In the local shops I've found only Core i7 with 6 cores (like Core
> > i7-39xx) and 4 cores. Should I obtain much better performance with 6
> cores
> > than with 4 cores in case of i7 cpu (assuming that I run simulation in
> > cpu+gpu mode )?
> >
> > Also you've mentioned about 4 PCeI MD. Does it means that modern
> > work-station could have 4 GPU's in one home-like desktop ? According to
>

Aw: Re: Re: [gmx-users] GPU-based workstation

[lloyd riggs]

 

You can also look at profilling on varied web sites, the high end Nvidia run only slightly better than the 2 year old ones, from an individual point not worth the money yet, but if you have the money? as I've been browsing.

 

Also, the sim I did on the cluster was 180-190,000 atoms so the exact same performance the other person had.

 

Stephan


Gesendet: Samstag, 25. Mai 2013 um 15:19 Uhr
Von: "James Starlight" 
An: "Discussion list for GROMACS users" 
Betreff: Re: Aw: Re: [gmx-users] GPU-based workstation

Richard,

thanks for suggestion!

Assuming that I'm using 2 high end GeForce's what performance be better

1) in case of one i7 (4 or 6 nodes ) ?

2) in case of 8 core Xeon like CPU Intel Xeon E5-2650 2.0 GHz / 8core

What properties of MB should take into account primarily for such
Xenon-based system. Does such MBs support multi-GPU ( I noticed that many
such MBs lack for PCI)?

James

2013/5/25 Broadbent, Richard 

> I've been running on my Universities GPU nodes these are one E5-xeon
> (6-cores 12 threads) and have 4 Nvidia 690gtx's. My system is 93 000 atoms
> of DMF under NVE. The performance has been a little disappointing
> ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I get
> 5.4ns/day for the same system. On our HPC system using 32 nodes each with 2
> quad-core xeon processors I get 30-40ns/day.
>
> I think that to achieve reasonable performance the system has to be
> balanced between CPU's and GPU's probably getting 2 high end GPU's and a
> top end xeon E5 or core i7 would be a good choice.
>
>
> Richard
>
> From: lloyd riggs >
> Reply-To: Discussion users 
> gmx-users@gromacs.org>>
> Date: Saturday, 25 May 2013 12:02
> To: Discussion users >
> Subject: Aw: Re: [gmx-users] GPU-based workstation
>
> More RAM the better, and the best I have seen is 4 GPU work station. I
> can use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
> really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8
> or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
> though with GPU's, as on a board there packed, so extra cooling things may
> help not blow a GPU, but I would look for good ones (ask around), as its a
> video game market and they go for looks even though its in casing? The
> external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont
> know performance wise, the onboard GPU, the more RAM the marrier...so yes,
> normal work stations you can get 4 GPU's for a 300 US$ board, but then the
> price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad
> is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> software, not Gromacs, so would be nice to see performance...for a small
> 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
> solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say
> 50 ps eq's....
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" 
> jmsstarli...@gmail.com>>
> An: "Discussion list for GROMACS users" 
> gmx-users@gromacs.org>>
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs >
>
> > There's also these, but 1 chip runs 6K US, they can get performance up to
> > 2.3 teraflops per chip though double percission...but have no clue about
> > integration with GPU's...Intell also sells their chips on PCIe
> cards...but
> > get only about 350 Gflops, and run 1K US$.
> >
> > http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
> > http://www.xilinx.com/
> >
> > They can design them though to fit a PCIe slot and run about the same,
> but
> > still need the board, ram etc...
> >
> > Mostly just 

Aw: Re: Re: [gmx-users] GPU-based workstation

[lloyd riggs]

 

Id go for the i7 6 core,

 

To the other message, funny.  I bought ATI's as they clock faster and cost 1/3 the price of Nvidia's but then the software all went to Nvidia.  The new ATI with twice the shaders runs at the same speed (around 1-1.3 terflops ) due to the same problems the Nvidias ran into with IO (or maybe onboard RAM does solve the problem if they went up to 16 or 32 MB)  Gromacs, etc...doesn't run on ATI's, and I've been hoping they, AMD,  catch up, but all I ever see is the constant in 6 months then nothing.

 

I ran around 40 4 ns simulations on University blades with 8 AMD quad cores, using 3 blades I only was able to get 1 ns/day, but never pressed it as far as why so slow, as I needed to finish.  With the Nvidia at  even 5 ns/day I or alot of people could do some really nice work as far as publishing, with raw data in 2 weeks time, so now I feel a bit saddened...

 

I also just found openCL profilling with CUDA 5 that will take any C or C++ software, and mark all sections you need to convert to openCL, but the trial software is 30 day, then 250 US$...

 

Stephan


Gesendet: Samstag, 25. Mai 2013 um 15:19 Uhr
Von: "James Starlight" 
An: "Discussion list for GROMACS users" 
Betreff: Re: Aw: Re: [gmx-users] GPU-based workstation

Richard,

thanks for suggestion!

Assuming that I'm using 2 high end GeForce's what performance be better

1) in case of one i7 (4 or 6 nodes ) ?

2) in case of 8 core Xeon like CPU Intel Xeon E5-2650 2.0 GHz / 8core

What properties of MB should take into account primarily for such
Xenon-based system. Does such MBs support multi-GPU ( I noticed that many
such MBs lack for PCI)?

James

2013/5/25 Broadbent, Richard 

> I've been running on my Universities GPU nodes these are one E5-xeon
> (6-cores 12 threads) and have 4 Nvidia 690gtx's. My system is 93 000 atoms
> of DMF under NVE. The performance has been a little disappointing
> ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I get
> 5.4ns/day for the same system. On our HPC system using 32 nodes each with 2
> quad-core xeon processors I get 30-40ns/day.
>
> I think that to achieve reasonable performance the system has to be
> balanced between CPU's and GPU's probably getting 2 high end GPU's and a
> top end xeon E5 or core i7 would be a good choice.
>
>
> Richard
>
> From: lloyd riggs >
> Reply-To: Discussion users 
> gmx-users@gromacs.org>>
> Date: Saturday, 25 May 2013 12:02
> To: Discussion users >
> Subject: Aw: Re: [gmx-users] GPU-based workstation
>
> More RAM the better, and the best I have seen is 4 GPU work station. I
> can use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is
> really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8
> or 10 slot board really holds 8 or 10 GPU's). There's cooling problems
> though with GPU's, as on a board there packed, so extra cooling things may
> help not blow a GPU, but I would look for good ones (ask around), as its a
> video game market and they go for looks even though its in casing? The
> external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont
> know performance wise, the onboard GPU, the more RAM the marrier...so yes,
> normal work stations you can get 4 GPU's for a 300 US$ board, but then the
> price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad
> is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> software, not Gromacs, so would be nice to see performance...for a small
> 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> minutes real for 1 ns sim, but tried simple large 800 amino, 25,000
> solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say
> 50 ps eq's
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight" 
> jmsstarli...@gmail.com>>
> An: "Discussion list for GROMACS users" 
> gmx-users@gromacs.org>>
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attrib

Re: Aw: Re: [gmx-users] GPU-based workstation

[James Starlight]

Richard,

thanks for suggestion!

Assuming that I'm using 2 high end GeForce's what performance be better

1) in case of one i7 (4 or 6 nodes ) ?

2) in case of 8 core Xeon like  CPU Intel Xeon E5-2650 2.0 GHz / 8core

What properties of MB should take into account primarily for such
Xenon-based system. Does such MBs support multi-GPU ( I noticed that many
such MBs lack for PCI)?

James

2013/5/25 Broadbent, Richard 

> I've been running on my Universities GPU nodes these are one E5-xeon
> (6-cores 12 threads)  and have 4 Nvidia 690gtx's. My system is 93 000 atoms
> of DMF under NVE.  The performance has been a little disappointing
> ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I get
> 5.4ns/day for the same system. On our HPC system using 32 nodes each with 2
> quad-core xeon processors I get 30-40ns/day.
>
> I think that to achieve reasonable performance the system has to be
> balanced between CPU's and GPU's probably getting 2 high end GPU's and a
> top end xeon E5 or core i7 would be a good choice.
>
>
> Richard
>
> From: lloyd riggs mailto:lloyd.ri...@gmx.ch>>
> Reply-To: Discussion users  gmx-users@gromacs.org>>
> Date: Saturday, 25 May 2013 12:02
> To: Discussion users mailto:gmx-users@gromacs.org>>
> Subject: Aw: Re: [gmx-users] GPU-based workstation
>
> More RAM the better, and the best I have seen is 4 GPU work station.  I
> can use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is
> really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8
> or 10 slot board really holds 8 or 10 GPU's).  There's cooling problems
> though with GPU's, as on a board there packed, so extra cooling things may
> help not blow a GPU, but I would look for good ones (ask around), as its a
> video game market and they go for looks even though its in casing?  The
> external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont
> know performance wise, the onboard GPU, the more RAM the marrier...so yes,
> normal work stations you can get 4 GPU's for a 300 US$ board, but then the
> price goes way up (3-4000 US$ for an 8-10 gpu board).  RAM ordered abroad
> is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests
> software, not Gromacs, so would be nice to see performance...for a small
> 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10
> minutes real  for 1 ns sim, but tried simple large 800 amino, 25,000
> solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say
> 50 ps eq's
>
> Stephan
>
> Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
> Von: "James Starlight"  jmsstarli...@gmail.com>>
> An: "Discussion list for GROMACS users"  gmx-users@gromacs.org>>
> Betreff: Re: [gmx-users] GPU-based workstation
> Dear Dr. Watkins!
>
> Thank you for the suggestions!
>
> In the local shops I've found only Core i7 with 6 cores (like Core
> i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
> than with 4 cores in case of i7 cpu (assuming that I run simulation in
> cpu+gpu mode )?
>
> Also you've mentioned about 4 PCeI MD. Does it means that modern
> work-station could have 4 GPU's in one home-like desktop ? According to my
> current task I suppose that 2 GPU's would be suitable for my simulations
> (assuming that I use typical ASUS MB and 650 Watt power unit). Have
> someone tried to use several GPU's on one workstation ? What attributes of
> MB should be taken into account for best performance on such multi-gpu
> station ?
>
> James
>
> 2013/5/25 lloyd riggs mailto:lloyd.ri...@gmx.ch>>
>
> > There's also these, but 1 chip runs 6K US, they can get performance up to
> > 2.3 teraflops per chip though double percission...but have no clue about
> > integration with GPU's...Intell also sells their chips on PCIe
> cards...but
> > get only about 350 Gflops, and run 1K US$.
> >
> > http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
> > http://www.xilinx.com/
> >
> > They can design them though to fit a PCIe slot and run about the same,
> but
> > still need the board, ram etc...
> >
> > Mostly just to dream about, they say you can order them with radiation
> > shielding as well...so...
> >
> > Stephan Watkins
> >
> > *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
> > *Von:* "James Starlight"  jmsstarli...@gmail.com>>
> > *An:* "Discussion list for GROMACS users"  gmx-users@gromacs.org>>
> > *Betreff:* [gmx-users] GPU-based workstation
> > Dear Gromacs Users!
>

Re: Aw: Re: [gmx-users] GPU-based workstation

[Broadbent, Richard]

I've been running on my Universities GPU nodes these are one E5-xeon (6-cores 
12 threads)  and have 4 Nvidia 690gtx's. My system is 93 000 atoms of DMF under 
NVE.  The performance has been a little disappointing ~10ns/day. On my home 
system using a core i5-2500 and a nvidia 560ti I get 5.4ns/day for the same 
system. On our HPC system using 32 nodes each with 2 quad-core xeon processors 
I get 30-40ns/day.

I think that to achieve reasonable performance the system has to be balanced 
between CPU's and GPU's probably getting 2 high end GPU's and a top end xeon E5 
or core i7 would be a good choice.


Richard

From: lloyd riggs mailto:lloyd.ri...@gmx.ch>>
Reply-To: Discussion users mailto:gmx-users@gromacs.org>>
Date: Saturday, 25 May 2013 12:02
To: Discussion users mailto:gmx-users@gromacs.org>>
Subject: Aw: Re: [gmx-users] GPU-based workstation

More RAM the better, and the best I have seen is 4 GPU work station.  I can 
use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is really 
3-4 GPU, except the tyan mentioned (there designed as blades so an 8 or 10 slot 
board really holds 8 or 10 GPU's).  There's cooling problems though with GPU's, 
as on a board there packed, so extra cooling things may help not blow a GPU, 
but I would look for good ones (ask around), as its a video game market and 
they go for looks even though its in casing?  The external RAM (not onboard GPU 
RAM) helps if you do a larger sim, but I dont know performance wise, the 
onboard GPU, the more RAM the marrier...so yes, normal work stations you can 
get 4 GPU's for a 300 US$ board, but then the price goes way up (3-4000 US$ for 
an 8-10 gpu board).  RAM ordered abroad is also cheep, 8 or 16 MB Vs. Shop...I 
have used 4 GPU's but only on tests software, not Gromacs, so would be nice to 
see performance...for a small 100 atom molecule and 500 solvent, using just the 
CPU I get it to run 5-10 minutes real  for 1 ns sim, but tried simple large 800 
amino, 25,000 solvent eq (NVT or NPT) runs and they clock at around 1 hour real 
for say 50 ps eq's

Stephan

Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
Von: "James Starlight" mailto:jmsstarli...@gmail.com>>
An: "Discussion list for GROMACS users" 
mailto:gmx-users@gromacs.org>>
Betreff: Re: [gmx-users] GPU-based workstation
Dear Dr. Watkins!

Thank you for the suggestions!

In the local shops I've found only Core i7 with 6 cores (like Core
i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
than with 4 cores in case of i7 cpu (assuming that I run simulation in
cpu+gpu mode )?

Also you've mentioned about 4 PCeI MD. Does it means that modern
work-station could have 4 GPU's in one home-like desktop ? According to my
current task I suppose that 2 GPU's would be suitable for my simulations
(assuming that I use typical ASUS MB and 650 Watt power unit). Have
someone tried to use several GPU's on one workstation ? What attributes of
MB should be taken into account for best performance on such multi-gpu
station ?

James

2013/5/25 lloyd riggs mailto:lloyd.ri...@gmx.ch>>

> There's also these, but 1 chip runs 6K US, they can get performance up to
> 2.3 teraflops per chip though double percission...but have no clue about
> integration with GPU's...Intell also sells their chips on PCIe cards...but
> get only about 350 Gflops, and run 1K US$.
>
> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
> http://www.xilinx.com/
>
> They can design them though to fit a PCIe slot and run about the same, but
> still need the board, ram etc...
>
> Mostly just to dream about, they say you can order them with radiation
> shielding as well...so...
>
> Stephan Watkins
>
> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
> *Von:* "James Starlight" 
> mailto:jmsstarli...@gmail.com>>
> *An:* "Discussion list for GROMACS users" 
> mailto:gmx-users@gromacs.org>>
> *Betreff:* [gmx-users] GPU-based workstation
> Dear Gromacs Users!
>
>
> I'd like to build new workstation for performing simulation on GPU with
> Gromacs 4.6 native cuda support.
> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
> and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
> with SD integrator)
>
>
> Now I'd like to build multi-gpu wokstation.
>
> My question - How much GPU would give me best performance on the typical
> home-like workstation. What algorithm of Ncidia GPU integration should I
> use (e.g SLI etc) ?
>
>
> Thanks for help,
>
>
> James
> --
> gmx-users mailing list gmx-users@gromacs.org<mailto:gmx-users@gromacs.org>
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archiv

Aw: Re: [gmx-users] GPU-based workstation

[lloyd riggs]

More RAM the better, and the best I have seen is 4 GPU work station.  I can use/have used 4.  The GPU takes 2 slots though, so a 7-8 PCIe board is really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 or 10 slot board really holds 8 or 10 GPU's).  There's cooling problems though with GPU's, as on a board there packed, so extra cooling things may help not blow a GPU, but I would look for good ones (ask around), as its a video game market and they go for looks even though its in casing?  The external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont know performance wise, the onboard GPU, the more RAM the marrier...so yes, normal work stations you can get 4 GPU's for a 300 US$ board, but then the price goes way up (3-4000 US$ for an 8-10 gpu board).  RAM ordered abroad is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests software, not Gromacs, so would be nice to see performance...for a small 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10 minutes real  for 1 ns sim, but tried simple large 800 amino, 25,000 solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps eq's

 

Stephan

 

Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr
Von: "James Starlight" 
An: "Discussion list for GROMACS users" 
Betreff: Re: [gmx-users] GPU-based workstation

Dear Dr. Watkins!

Thank you for the suggestions!

In the local shops I've found only Core i7 with 6 cores (like Core
i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores
than with 4 cores in case of i7 cpu (assuming that I run simulation in
cpu+gpu mode )?

Also you've mentioned about 4 PCeI MD. Does it means that modern
work-station could have 4 GPU's in one home-like desktop ? According to my
current task I suppose that 2 GPU's would be suitable for my simulations
(assuming that I use typical ASUS MB and 650 Watt power unit). Have
someone tried to use several GPU's on one workstation ? What attributes of
MB should be taken into account for best performance on such multi-gpu
station ?

James

2013/5/25 lloyd riggs 

> There's also these, but 1 chip runs 6K US, they can get performance up to
> 2.3 teraflops per chip though double percission...but have no clue about
> integration with GPU's...Intell also sells their chips on PCIe cards...but
> get only about 350 Gflops, and run 1K US$.
>
> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
> http://www.xilinx.com/
>
> They can design them though to fit a PCIe slot and run about the same, but
> still need the board, ram etc...
>
> Mostly just to dream about, they say you can order them with radiation
> shielding as well...so...
>
> Stephan Watkins
>
> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
> *Von:* "James Starlight" 
> *An:* "Discussion list for GROMACS users" 
> *Betreff:* [gmx-users] GPU-based workstation
> Dear Gromacs Users!
>
>
> I'd like to build new workstation for performing simulation on GPU with
> Gromacs 4.6 native cuda support.
> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
> and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
> with SD integrator)
>
>
> Now I'd like to build multi-gpu wokstation.
>
> My question - How much GPU would give me best performance on the typical
> home-like workstation. What algorithm of Ncidia GPU integration should I
> use (e.g SLI etc) ?
>
>
> Thanks for help,
>
>
> James
> --
> gmx-users mailing list gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gmx-users mailing list gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing list gmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU-based workstation

[James Starlight]

Dear Dr. Watkins!

Thank you for the suggestions!

In the local shops I've found only Core i7 with 6 cores (like  Core
i7-39xx) and 4 cores.  Should I obtain much better performance with 6 cores
than with 4 cores in case of i7 cpu (assuming that I run simulation in
cpu+gpu mode )?

Also you've mentioned about 4 PCeI MD. Does it means that modern
work-station could have 4 GPU's in one home-like desktop ? According to my
current task I suppose that 2 GPU's would be suitable for my simulations
(assuming that I use typical ASUS MB  and 650 Watt power unit). Have
someone tried to use several GPU's on one workstation ? What attributes of
MB should be taken into account for best performance on such multi-gpu
station ?

James

2013/5/25 lloyd riggs 

> There's also these, but 1 chip runs 6K US, they can get performance up to
> 2.3 teraflops per chip though double percission...but have no clue about
> integration with GPU's...Intell also sells their chips on PCIe cards...but
> get only about 350 Gflops, and run 1K US$.
>
> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor
> http://www.xilinx.com/
>
> They can design them though to fit a PCIe slot and run about the same, but
> still need the board, ram etc...
>
> Mostly just to dream about, they say you can order them with radiation
> shielding as well...so...
>
> Stephan Watkins
>
> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr
> *Von:* "James Starlight" 
> *An:* "Discussion list for GROMACS users" 
> *Betreff:* [gmx-users] GPU-based workstation
> Dear Gromacs Users!
>
>
> I'd like to build new workstation for performing simulation on GPU with
> Gromacs 4.6 native cuda support.
> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
> and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
> with SD integrator)
>
>
> Now I'd like to build multi-gpu wokstation.
>
> My question - How much GPU would give me best performance on the typical
> home-like workstation. What algorithm of Ncidia GPU integration should I
> use (e.g SLI etc) ?
>
>
> Thanks for help,
>
>
> James
> --
> gmx-users mailing list gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Aw: [gmx-users] GPU-based workstation

[lloyd riggs]

There's also these, but 1 chip runs 6K US, they can get performance up to 2.3 teraflops per chip though double percission...but have no clue about integration with GPU's...Intell also sells their chips on PCIe cards...but get only about 350 Gflops, and run 1K US$.

 

http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor

http://www.xilinx.com/

 

They can design them though to fit a PCIe slot and run about the same, but still need the board, ram etc...

 

Mostly just to dream about, they say you can order them with radiation shielding as well...so...

 

Stephan Watkins

 

Gesendet: Freitag, 24. Mai 2013 um 13:17 Uhr
Von: "James Starlight" 
An: "Discussion list for GROMACS users" 
Betreff: [gmx-users] GPU-based workstation

Dear Gromacs Users!


I'd like to build new workstation for performing simulation on GPU with
Gromacs 4.6 native cuda support.
Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
with SD integrator)


Now I'd like to build multi-gpu wokstation.

My question - How much GPU would give me best performance on the typical
home-like workstation. What algorithm of Ncidia GPU integration should I
use (e.g SLI etc) ?


Thanks for help,


James
--
gmx-users mailing list gmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists



-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Aw: [gmx-users] GPU-based workstation

[lloyd riggs]

Dear Dr. Starlight,

 

Dont know the answere to all, but funny I was looking at performance on varied others web sites.  I use a core i7 970, but it seems their newest chip is almost the same as the i7 in performance (thier newer chips dont scale past 12 cores for some internal chip based design, they put out a 60 core chip for 1K and it doesnt even reach 1 terflop), and the newer GPU's suffer from board IO limits due the PCIe being maxed out.

 

As far as boards, home PC (im in Switzerland so have been mind distroyed as far as prices), there's a 4 CPU 10 GPU tyan board for 4K US (if you dig enough maybe 3,500) which with ram, chips and a single Nvidia would run up to 5-6 K US$.  The i7 970's can be baught for 30-50 US$ a pop now if you look becuase intel overshot its abilities and gave off the rights to several bulk manufacturers (all LG 1366 socket chips even i5's) as they add more pins (2100 or something), but you have to spend a day or two on the internet looking through sellers abroad.  The Nvida's are the pricy thing, however from what I have read, you dont get much more from their latest as well as mentioned, so 2 more years waiting, or something like I mentioned with a 2 year old Nvidia or two, would be the best especially for research purposes.  Otherwise a single 300 US board with 4 usable PCIe's (ASUS arnt bad), ram, 1 i7 970, and 1 Nvida from 2 years back gives you around 1.3 terflops, with their lates 1.4-1.5 teraflops (integrated use of chip alone).  The i7 I have I have got around 350 Gflops for some things with 8 threads.  I use radeons though, although this requires user programming, and if you use ther FFT libraries, you have to sign your first child over and loose any freeware redistribution rights in any way, otherwise their supposedly supposed to have all the same CUDA like libraries by the end of the summer (Portlandgroup , openCL), but I heard the same for 2 years now.

 

Sincerely,

 

Stephan Watkins

 

Gesendet: Freitag, 24. Mai 2013 um 13:17 Uhr
Von: "James Starlight" 
An: "Discussion list for GROMACS users" 
Betreff: [gmx-users] GPU-based workstation

Dear Gromacs Users!


I'd like to build new workstation for performing simulation on GPU with
Gromacs 4.6 native cuda support.
Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video
and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
with SD integrator)


Now I'd like to build multi-gpu wokstation.

My question - How much GPU would give me best performance on the typical
home-like workstation. What algorithm of Ncidia GPU integration should I
use (e.g SLI etc) ?


Thanks for help,


James
--
gmx-users mailing list gmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists



-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU-based workstation

[James Starlight]

Dear Gromacs Users!


I'd like to build new workstation for performing simulation on GPU with
Gromacs 4.6 native cuda support.
Recently I've used such setup with  Core i5 cpu and nvidia 670 GTX video
and obtain good performance ( ~ 20 ns\day for typical 60.000 atom system
with SD integrator)


Now I'd like to build multi-gpu wokstation.

My question - How much GPU would give me best performance on the typical
home-like workstation. What algorithm of Ncidia GPU integration should I
use (e.g SLI etc) ?


Thanks for help,


James
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Albert]

the problem is still there...

:-(



On 04/29/2013 06:06 PM, Szilárd Páll wrote:

On Mon, Apr 29, 2013 at 3:51 PM, Albert  wrote:

>On 04/29/2013 03:47 PM, Szilárd Páll wrote:

>>
>>In that case, while it isn't very likely, the issue could be caused by
>>some implementation detail which aims to avoid performance loss caused
>>by an issue in the NVIDIA drivers.
>>
>>Try running with the GMX_CUDA_STREAMSYNC environment variable set.
>>
>>Btw, were there any other processes using the GPU while mdrun was running?
>>
>>Cheers,
>>--
>>Szilárd

>
>
>thanks for kind reply.
>There is no any other process when I am running Gromacs.
>
>do you mean I should set GMX_CUDA_STREAMSYNC in the job script like:
>
>export GMX_CUDA_STREAMSYNC=/opt/cuda-5.0

Sort of, but the value does not matter. So if your shell is bash, the
above as well as simply "export GMX_CUDA_STREAMSYNC=" will work fine.

Let us know if this avoided the crash - when you have simulated long
enough to be able to judge.

Cheers,
--
Szilárd



--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Szilárd Páll]

On Mon, Apr 29, 2013 at 3:51 PM, Albert  wrote:
> On 04/29/2013 03:47 PM, Szilárd Páll wrote:
>>
>> In that case, while it isn't very likely, the issue could be caused by
>> some implementation detail which aims to avoid performance loss caused
>> by an issue in the NVIDIA drivers.
>>
>> Try running with the GMX_CUDA_STREAMSYNC environment variable set.
>>
>> Btw, were there any other processes using the GPU while mdrun was running?
>>
>> Cheers,
>> --
>> Szilárd
>
>
> thanks for kind reply.
> There is no any other process when I am running Gromacs.
>
> do you mean I should set GMX_CUDA_STREAMSYNC in the job script like:
>
> export GMX_CUDA_STREAMSYNC=/opt/cuda-5.0

Sort of, but the value does not matter. So if your shell is bash, the
above as well as simply "export GMX_CUDA_STREAMSYNC=" will work fine.

Let us know if this avoided the crash - when you have simulated long
enough to be able to judge.

Cheers,
--
Szilárd

>
> ?
>
> THX
> Albert
>
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Albert]

On 04/29/2013 03:47 PM, Szilárd Páll wrote:

In that case, while it isn't very likely, the issue could be caused by
some implementation detail which aims to avoid performance loss caused
by an issue in the NVIDIA drivers.

Try running with the GMX_CUDA_STREAMSYNC environment variable set.

Btw, were there any other processes using the GPU while mdrun was running?

Cheers,
--
Szilárd


thanks for kind reply.
There is no any other process when I am running Gromacs.

do you mean I should set GMX_CUDA_STREAMSYNC in the job script like:

export GMX_CUDA_STREAMSYNC=/opt/cuda-5.0

?

THX
Albert



--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Szilárd Páll]

In that case, while it isn't very likely, the issue could be caused by
some implementation detail which aims to avoid performance loss caused
by an issue in the NVIDIA drivers.

Try running with the GMX_CUDA_STREAMSYNC environment variable set.

Btw, were there any other processes using the GPU while mdrun was running?

Cheers,
--
Szilárd


On Mon, Apr 29, 2013 at 3:32 PM, Albert  wrote:
> On 04/29/2013 03:31 PM, Szilárd Páll wrote:
>>
>> The segv indicates that mdrun crashed and not that the machine was
>> restarted. The GPU detection output (both on stderr and log) should
>> show whether ECC is "on" (and so does the nvidia-smi tool).
>>
>> Cheers,
>> --
>> Szilárd
>
>
> yes it was on:
>
>
> Reading file heavy.tpr, VERSION 4.6.1 (single precision)
> Using 4 MPI threads
> Using 8 OpenMP threads per tMPI thread
>
> 5 GPUs detected:
>   #0: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
>   #1: NVIDIA GeForce GTX 650, compute cap.: 3.0, ECC:  no, stat: compatible
>   #2: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
>   #3: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
>   #4: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
>
> 4 GPUs user-selected for this run: #0, #2, #3, #4
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Albert]

On 04/29/2013 03:31 PM, Szilárd Páll wrote:

The segv indicates that mdrun crashed and not that the machine was
restarted. The GPU detection output (both on stderr and log) should
show whether ECC is "on" (and so does the nvidia-smi tool).

Cheers,
--
Szilárd


yes it was on:


Reading file heavy.tpr, VERSION 4.6.1 (single precision)
Using 4 MPI threads
Using 8 OpenMP threads per tMPI thread

5 GPUs detected:
  #0: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
  #1: NVIDIA GeForce GTX 650, compute cap.: 3.0, ECC:  no, stat: compatible
  #2: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
  #3: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible
  #4: NVIDIA Tesla K20m, compute cap.: 3.5, ECC: yes, stat: compatible

4 GPUs user-selected for this run: #0, #2, #3, #4

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Szilárd Páll]

On Mon, Apr 29, 2013 at 2:41 PM, Albert  wrote:
> On 04/28/2013 05:45 PM, Justin Lemkul wrote:
>>
>>
>> Frequent failures suggest instability in the simulated system. Check your
>> .log file or stderr for informative Gromacs diagnostic information.
>>
>> -Justin
>
>
>
> my log file didn't have any errors, the end of topped log file something
> like:
>
> DD  step 2259  vol min/aver 0.967  load imb.: force  0.8%
>
>Step   Time Lambda
>226045200.00.0
>
>Energies (kJ/mol)
>   AngleU-BProper Dih.  Improper Dih.  LJ-14
> 9.86437e+034.02406e+043.52809e+046.13542e+02 8.61815e+03
>  Coulomb-14LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.
> 1.25055e+043.05477e+04   -9.05956e+03   -6.02400e+05 1.58357e+03
>  Position Rest.  PotentialKinetic En.   Total Energy Temperature
> 1.39149e+02   -4.72066e+051.37165e+05   -3.34901e+05 3.11958e+02
>  Pres. DC (bar) Pressure (bar)   Constr. rmsd
>-2.94092e+02   -7.91535e+011.79812e-05
>
>
> also in the information file I only obtained information:
>
>
> step 13300, will finish Tue Apr 30 14:41
> NOTE: Turning on dynamic load balancing
>
>
> Probably the machine was restarted from time to time?

The segv indicates that mdrun crashed and not that the machine was
restarted. The GPU detection output (both on stderr and log) should
show whether ECC is "on" (and so does the nvidia-smi tool).

Cheers,
--
Szilárd


>
> best
> Albert
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Albert]

On 04/28/2013 05:45 PM, Justin Lemkul wrote:


Frequent failures suggest instability in the simulated system. Check 
your .log file or stderr for informative Gromacs diagnostic information.


-Justin 



my log file didn't have any errors, the end of topped log file something 
like:


DD  step 2259  vol min/aver 0.967  load imb.: force  0.8%

   Step   Time Lambda
   226045200.00.0

   Energies (kJ/mol)
  AngleU-BProper Dih.  Improper Dih.  LJ-14
9.86437e+034.02406e+043.52809e+046.13542e+02 8.61815e+03
 Coulomb-14LJ (SR)  Disper. corr.   Coulomb (SR)   Coul. recip.
1.25055e+043.05477e+04   -9.05956e+03   -6.02400e+05 1.58357e+03
 Position Rest.  PotentialKinetic En.   Total Energy Temperature
1.39149e+02   -4.72066e+051.37165e+05   -3.34901e+05 3.11958e+02
 Pres. DC (bar) Pressure (bar)   Constr. rmsd
   -2.94092e+02   -7.91535e+011.79812e-05


also in the information file I only obtained information:


step 13300, will finish Tue Apr 30 14:41
NOTE: Turning on dynamic load balancing


Probably the machine was restarted from time to time?

best
Albert


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Albert]

Hello:

 yes, I tried the CPU only version, it goes well and didn't stop. I am 
not sure whether I have ECC on or not. There are 4 Tesla K20 and one 
GTX650 in the workstation, after compilation, I simple submit the jobs 
with command:



mdrun -s md.tpr -gpu_id 0234

I submit the same system in another GTX690 machine, it also goes 
well. I compiled Gromacs with the same options in that machine.


thank you very much
best
Albert



On 04/29/2013 01:19 PM, Szilárd Páll wrote:

Have you tried running on CPUs only just to see if the issue persists?
Unless the issue does not occur with the same binary on the same
hardware running on CPUs only, I doubt it's a problem in the code.

Do you have ECC on?
--
Szilárd


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Szilárd Páll]

Have you tried running on CPUs only just to see if the issue persists?
Unless the issue does not occur with the same binary on the same
hardware running on CPUs only, I doubt it's a problem in the code.

Do you have ECC on?
--
Szilárd


On Sun, Apr 28, 2013 at 5:27 PM, Albert  wrote:
> Dear:
>
>   I am running MD jobs in a workstation with 4 K20 GPU and I found that the
> job always failed with following messages from time to time:
>
>
> [tesla:03432] *** Process received signal ***
> [tesla:03432] Signal: Segmentation fault (11)
> [tesla:03432] Signal code: Address not mapped (1)
> [tesla:03432] Failing at address: 0xfffe02de67e0
> [tesla:03432] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)
> [0x7f4666da1cb0]
> [tesla:03432] [ 1] mdrun_mpi() [0x47dd61]
> [tesla:03432] [ 2] mdrun_mpi() [0x47d8ae]
> [tesla:03432] [ 3]
> /opt/intel/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93)
> [0x7f46667904f3]
> [tesla:03432] *** End of error message ***
> --
> mpirun noticed that process rank 0 with PID 3432 on node tesla exited on
> signal 11 (Segmentation fault).
> --
>
>
> I can continue the jobs with mdrun option "-append -cpi", but it still
> stopped from time to time. I am just wondering what's the problem?
>
> thank you very much
> Albert
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU job often stopped

[Justin Lemkul]

On 4/28/13 11:27 AM, Albert wrote:

Dear:

   I am running MD jobs in a workstation with 4 K20 GPU and I found that the job
always failed with following messages from time to time:


[tesla:03432] *** Process received signal ***
[tesla:03432] Signal: Segmentation fault (11)
[tesla:03432] Signal code: Address not mapped (1)
[tesla:03432] Failing at address: 0xfffe02de67e0
[tesla:03432] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) 
[0x7f4666da1cb0]
[tesla:03432] [ 1] mdrun_mpi() [0x47dd61]
[tesla:03432] [ 2] mdrun_mpi() [0x47d8ae]
[tesla:03432] [ 3]
/opt/intel/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93) [0x7f46667904f3]
[tesla:03432] *** End of error message ***
--
mpirun noticed that process rank 0 with PID 3432 on node tesla exited on signal
11 (Segmentation fault).
--


I can continue the jobs with mdrun option "-append -cpi", but it still stopped
from time to time. I am just wondering what's the problem?



Frequent failures suggest instability in the simulated system.  Check your .log 
file or stderr for informative Gromacs diagnostic information.


-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU job often stopped

[Albert]

Dear:

  I am running MD jobs in a workstation with 4 K20 GPU and I found that 
the job always failed with following messages from time to time:



[tesla:03432] *** Process received signal ***
[tesla:03432] Signal: Segmentation fault (11)
[tesla:03432] Signal code: Address not mapped (1)
[tesla:03432] Failing at address: 0xfffe02de67e0
[tesla:03432] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) 
[0x7f4666da1cb0]

[tesla:03432] [ 1] mdrun_mpi() [0x47dd61]
[tesla:03432] [ 2] mdrun_mpi() [0x47d8ae]
[tesla:03432] [ 3] 
/opt/intel/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93) 
[0x7f46667904f3]

[tesla:03432] *** End of error message ***
--
mpirun noticed that process rank 0 with PID 3432 on node tesla exited on 
signal 11 (Segmentation fault).

--


I can continue the jobs with mdrun option "-append -cpi", but it still 
stopped from time to time. I am just wondering what's the problem?


thank you very much
Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU efficiency question

[Mark Abraham]

Probably the part of the calculation done on the GPU is not rate limiting.
There's no point having four chefs to make one dish...

Look at the beginning and end of your .log files for diagnostic
information. If this is a single node, you should be using threadMPI, not
real MPI. Generally four CPU cores vs four GPU cores will require an
extremely large PP load for the GPUs to all be effective.

Mark


On Fri, Apr 26, 2013 at 8:35 PM, Albert  wrote:

> Dear:
>
>  I've got two GTX690 in a a workstation and I found that when I run the md
> production with following two command:
>
> mpirun -np 4 md_run_mpi
>
> or
>
> mpirun -np 2 md_run_mpi
>
> the efficiency are the same. I notice that gromacs can detect 4 GPU
> (probably because GTX690 have two core..):
>
> 4 GPUs detected on host node4:
>   #0: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>   #1: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>   #2: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>   #3: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
>
>
> why the "-np 2" and "-np 4" are the same efficiency? shouldn't it be
> faster for "-np 4" ?
>
> thank you very much
>
> Albert
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU efficiency question

[Albert]

Dear:

 I've got two GTX690 in a a workstation and I found that when I run the 
md production with following two command:


mpirun -np 4 md_run_mpi

or

mpirun -np 2 md_run_mpi

the efficiency are the same. I notice that gromacs can detect 4 GPU 
(probably because GTX690 have two core..):


4 GPUs detected on host node4:
  #0: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible
  #3: NVIDIA GeForce GTX 690, compute cap.: 3.0, ECC:  no, stat: compatible


why the "-np 2" and "-np 4" are the same efficiency? shouldn't it be 
faster for "-np 4" ?


thank you very much

Albert

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU performance

[Szilárd Páll]

On Wed, Apr 10, 2013 at 3:34 AM, Benjamin Bobay  wrote:

> Szilárd -
>
> First, many thanks for the reply.
>
> Second, I am glad that I am not crazy.
>
> Ok so based on your suggestions, I think I know what the problem is/was.
> There was a sander process running on 1 of the CPUs.  Clearly GROMACS was
> trying to use 4 with "Using 4 OpenMP thread". I just did not catch that.
> Sorry! Rookie mistake.
>
> Which I guess leads me to my next question (sorry if its too naive):
>
> (1) When running GROMACS (or a I guess any other CUDA based programs), its
> best to have all the CPUs free, right? I guess based on my results I have
> pretty much answered that question.  Although I thought that as long as I
> have one CPU available to run the GPU it would be good: would setting
> "-ntmpi 1 -ntomp 1" help or would I take a major hit in ns/day as well?
>

Such a behavior is not specific to GROMACS or CUDA-accelerated codes, but
all compute-intensive codes that expect to be running "alone" on the set of
CPU cores they are started on. As you could see on the output, mdrun
automatically detected that you have 4 CPU cores and as Mark saied, it
tries to use all of them along the GPU. As one of the cores was busy, you
ended up in a situation in which four threads of mdrun plus the
(presumably) one thread of sander are competing for four cores. This is
made even worse by the fact that when using a full machine, mdrun locks its
threads to physical cores to prevent the OS from moving them around (which
can cause performance loss).

Secondly, using a single core with a GPU will not result in a very good
performance in GROMACS. The current GROMACS acceleration expects to run on
a couple of CPU cores together with a GPU - which is the typical balance of
CPU-GPU hardware most clusters (1 GPU/socket) as well as many home users
would have (1-2 GPUs for 4-8 CPU cores).


>
> If I try the benchmarks again just to see (for fun) with "Using 4 OpenMP
> thread", under top I have - so I think the CPU is fine :
> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
> 24791 bobayb20   0 48.3g  51m 7576 R 299.1  0.2  11:32.90
> mdrun
>

Nope, that just means, roughly speaking, that sander is probably fully
using one core and the four thread of mdrun are "crammed" on the remaining
three cores - which is bad.

However, you can simply run mdrun using three threads which will run fine
along sander. Whether this will be efficient or not, you'll have to see.
Note that if some other program is using the GPU as well, don't expect full
performance - but the difference will be much less than in the case
of oversubscribed CPU cores.

Cheers,
--
Szilárd


>
> When I have a chance (after this sander run is done - hopefully soon) I can
> try the benchmarks again.
>
> Thanks again for the help!
>
> Ben
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU performance

[Mark Abraham]

On Apr 10, 2013 3:34 AM, "Benjamin Bobay"  wrote:
>
> Szilárd -
>
> First, many thanks for the reply.
>
> Second, I am glad that I am not crazy.
>
> Ok so based on your suggestions, I think I know what the problem is/was.
> There was a sander process running on 1 of the CPUs.  Clearly GROMACS was
> trying to use 4 with "Using 4 OpenMP thread". I just did not catch that.
> Sorry! Rookie mistake.
>
> Which I guess leads me to my next question (sorry if its too naive):
>
> (1) When running GROMACS (or a I guess any other CUDA based programs), its
> best to have all the CPUs free, right? I guess based on my results I have
> pretty much answered that question.  Although I thought that as long as I
> have one CPU available to run the GPU it would be good: would setting
> "-ntmpi 1 -ntomp 1" help or would I take a major hit in ns/day as well?

Some codes might treat the CPU as a "I/O, MPI and memory-serving
co-processor" of the GPU; those codes will tend to be insensitive to the
CPU config. GROMACS goes to great lengths to use all the hardware in a
dynamically load-balanced way, so CPU load and config tend to affect the
bottom line immediately.

Mark

> If I try the benchmarks again just to see (for fun) with "Using 4 OpenMP
> thread", under top I have - so I think the CPU is fine :
> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
> 24791 bobayb20   0 48.3g  51m 7576 R 299.1  0.2  11:32.90
> mdrun
>
>
> When I have a chance (after this sander run is done - hopefully soon) I
can
> try the benchmarks again.
>
> Thanks again for the help!
>
> Ben
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU performance

[Benjamin Bobay]

Szilárd -

First, many thanks for the reply.

Second, I am glad that I am not crazy.

Ok so based on your suggestions, I think I know what the problem is/was.
There was a sander process running on 1 of the CPUs.  Clearly GROMACS was
trying to use 4 with "Using 4 OpenMP thread". I just did not catch that.
Sorry! Rookie mistake.

Which I guess leads me to my next question (sorry if its too naive):

(1) When running GROMACS (or a I guess any other CUDA based programs), its
best to have all the CPUs free, right? I guess based on my results I have
pretty much answered that question.  Although I thought that as long as I
have one CPU available to run the GPU it would be good: would setting
"-ntmpi 1 -ntomp 1" help or would I take a major hit in ns/day as well?

If I try the benchmarks again just to see (for fun) with "Using 4 OpenMP
thread", under top I have - so I think the CPU is fine :
PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
24791 bobayb20   0 48.3g  51m 7576 R 299.1  0.2  11:32.90
mdrun


When I have a chance (after this sander run is done - hopefully soon) I can
try the benchmarks again.

Thanks again for the help!

Ben
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU performance

[Szilárd Páll]

Hi Ben,

That performance is not reasonable at all - neither for CPU only run on
your quad-core Sandy Bridge, nor for the CPU+GPU run. For the latter you
should be getting more like 50 ns/day or so.

What's strange about your run is that the CPU-GPU load balancing is picking
a *very* long cut-off which means that your CPU is for some reason
performing very badly. Check how is mdrun behaving while running in
top/htop nad if you are not seeing ~400% CPU utilization, there is
something wrong - perhaps threads getting locked to the same core (to check
that try -pin off).

Secondly, note that you are using OpenMM-specific settings from the old
GROMACS-OpenMM comparison benchmarks in which the grid spacing is overly
coarse (you could use something like a fourier-spacing=0.125 or even larger
with rc=1.0).

Cheers,

--
Szilárd


On Tue, Apr 9, 2013 at 10:27 PM, Benjamin Bobay  wrote:

> Good afternoon -
>
> I recently installed gromacs-4.6 on CentOS6.3 and the installation went
> just fine.
>
> I have a Tesla C2075 GPU.
>
> I then downloaded the benchmark directories and ran a bench mark on the
> GPU/ dhfr-solv-PME.bench
>
> This is what I got:
>
> Using 1 MPI thread
> Using 4 OpenMP threads
>
> 1 GPU detected:
>   #0: NVIDIA Tesla C2075, compute cap.: 2.0, ECC: yes, stat: compatible
>
> 1 GPU user-selected for this run: #0
>
>
> Back Off! I just backed up ener.edr to ./#ener.edr.1#
> starting mdrun 'Protein in water'
> -1 steps, infinite ps.
> step   40: timed with pme grid 64 64 64, coulomb cutoff 1.000: 4122.9
> M-cycles
> step   80: timed with pme grid 56 56 56, coulomb cutoff 1.143: 3685.9
> M-cycles
> step  120: timed with pme grid 48 48 48, coulomb cutoff 1.333: 3110.8
> M-cycles
> step  160: timed with pme grid 44 44 44, coulomb cutoff 1.455: 3365.1
> M-cycles
> step  200: timed with pme grid 40 40 40, coulomb cutoff 1.600: 3499.0
> M-cycles
> step  240: timed with pme grid 52 52 52, coulomb cutoff 1.231: 3982.2
> M-cycles
> step  280: timed with pme grid 48 48 48, coulomb cutoff 1.333: 3129.2
> M-cycles
> step  320: timed with pme grid 44 44 44, coulomb cutoff 1.455: 3425.4
> M-cycles
> step  360: timed with pme grid 42 42 42, coulomb cutoff 1.524: 2979.1
> M-cycles
>   optimal pme grid 42 42 42, coulomb cutoff 1.524
> step 4300 performance: 1.8 ns/day
>
> and from the nvidia-smi output:
> Tue Apr  9 10:13:46 2013
> +--+
>
> | NVIDIA-SMI 4.304.37   Driver Version: 304.37
> |
>
> |---+--+--+
> | GPU  Name | Bus-IdDisp.  | Volatile Uncorr.
> ECC |
> | Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute
> M. |
>
> |===+==+==|
> |   0  Tesla C2075  | :03:00.0  On |
> 0 |
> | 30%   67CP080W / 225W |   4%  200MB / 5375MB |  4%
> Default |
>
> +---+--+--+
>
>
>
> +-+
> | Compute processes:   GPU
> Memory |
> |  GPU   PID  Process name
> Usage  |
>
> |=|
> |0 22568  mdrun
> 59MB  |
>
> +-+
>
>
> So I am only getting 1.8ns/day ! Is that right? It seems very very
> small compared to the CPU test where I am getting the same:
>
> step 200 performance: 1.8 ns/dayvol 0.79  imb F 14%
>
> >From the md.log of the GPU test:
> Detecting CPU-specific acceleration.
> Present hardware specification:
> Vendor: GenuineIntel
> Brand:  Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz
> Family:  6  Model: 45  Stepping:  7
> Features: aes apic avx clfsh cmov cx8 cx16 htt lahf_lm mmx msr nonstop_tsc
> pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 ssse3
> tdt x2a
> pic
> Acceleration most likely to fit this hardware: AVX_256
> Acceleration selected at GROMACS compile time: AVX_256
>
>
> 1 GPU detected:
>   #0: NVIDIA Tesla C2075, compute cap.: 2.0, ECC: yes, stat: compatible
>
> 1 GPU user-selected for this run: #0
>
> Will do PME sum in reciprocal space.
>
> Any thoughts as to why it is so slow?
>
> many thanks!
> Ben
>
> --
> 
> Research Assistant Professor
> North Carolina State University
> Department of Molecular and Structural Biochemistry
> 128 Polk Hall
> Raleigh, NC 27695
> Phone: (919)-513-0698
> Fax: (919)-515-2047
> 
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the

[gmx-users] GPU performance

[Benjamin Bobay]

Good afternoon -

I recently installed gromacs-4.6 on CentOS6.3 and the installation went
just fine.

I have a Tesla C2075 GPU.

I then downloaded the benchmark directories and ran a bench mark on the
GPU/ dhfr-solv-PME.bench

This is what I got:

Using 1 MPI thread
Using 4 OpenMP threads

1 GPU detected:
  #0: NVIDIA Tesla C2075, compute cap.: 2.0, ECC: yes, stat: compatible

1 GPU user-selected for this run: #0


Back Off! I just backed up ener.edr to ./#ener.edr.1#
starting mdrun 'Protein in water'
-1 steps, infinite ps.
step   40: timed with pme grid 64 64 64, coulomb cutoff 1.000: 4122.9
M-cycles
step   80: timed with pme grid 56 56 56, coulomb cutoff 1.143: 3685.9
M-cycles
step  120: timed with pme grid 48 48 48, coulomb cutoff 1.333: 3110.8
M-cycles
step  160: timed with pme grid 44 44 44, coulomb cutoff 1.455: 3365.1
M-cycles
step  200: timed with pme grid 40 40 40, coulomb cutoff 1.600: 3499.0
M-cycles
step  240: timed with pme grid 52 52 52, coulomb cutoff 1.231: 3982.2
M-cycles
step  280: timed with pme grid 48 48 48, coulomb cutoff 1.333: 3129.2
M-cycles
step  320: timed with pme grid 44 44 44, coulomb cutoff 1.455: 3425.4
M-cycles
step  360: timed with pme grid 42 42 42, coulomb cutoff 1.524: 2979.1
M-cycles
  optimal pme grid 42 42 42, coulomb cutoff 1.524
step 4300 performance: 1.8 ns/day

and from the nvidia-smi output:
Tue Apr  9 10:13:46 2013
+--+

| NVIDIA-SMI 4.304.37   Driver Version: 304.37
|
|---+--+--+
| GPU  Name | Bus-IdDisp.  | Volatile Uncorr.
ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute
M. |
|===+==+==|
|   0  Tesla C2075  | :03:00.0  On |
0 |
| 30%   67CP080W / 225W |   4%  200MB / 5375MB |  4%
Default |
+---+--+--+


+-+
| Compute processes:   GPU
Memory |
|  GPU   PID  Process name
Usage  |
|=|
|0 22568  mdrun
59MB  |
+-+


So I am only getting 1.8ns/day ! Is that right? It seems very very
small compared to the CPU test where I am getting the same:

step 200 performance: 1.8 ns/dayvol 0.79  imb F 14%

>From the md.log of the GPU test:
Detecting CPU-specific acceleration.
Present hardware specification:
Vendor: GenuineIntel
Brand:  Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz
Family:  6  Model: 45  Stepping:  7
Features: aes apic avx clfsh cmov cx8 cx16 htt lahf_lm mmx msr nonstop_tsc
pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 ssse3
tdt x2a
pic
Acceleration most likely to fit this hardware: AVX_256
Acceleration selected at GROMACS compile time: AVX_256


1 GPU detected:
  #0: NVIDIA Tesla C2075, compute cap.: 2.0, ECC: yes, stat: compatible

1 GPU user-selected for this run: #0

Will do PME sum in reciprocal space.

Any thoughts as to why it is so slow?

many thanks!
Ben

-- 

Research Assistant Professor
North Carolina State University
Department of Molecular and Structural Biochemistry
128 Polk Hall
Raleigh, NC 27695
Phone: (919)-513-0698
Fax: (919)-515-2047

-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[George Patargias]

Hi Szilard

Thanks for this tip; it was extremely useful. The problem was indeed the
incompatibility between the installed NVIDIA driver and the CUDA 5.0
runtime library. Installation of an older driver solved the problem. The
programs devideQuery etc can now detect the GPU.

GROMACS can also detect now the card but unfortunately aborts with the
following error

Fatal error: Incorrect launch configuration: mismatching number of PP MPI
processes and GPUs per node.
mdrun_mpi was started with 12 PP MPI processes per node, but only 1 GPU
were detected.

Here is my command line
mpirun -np 12 mdrun_mpi -s test.tpr -deffnm test_out -nb gpu

What can be the problem?

Thanks again

> Hi George,
> As I said before, that just means that most probably the GPU driver is
not
> compatible with the CUDA runtime (libcudart) that you installed with the
CUDA toolkit. I've no clue about the Mac OS installers and releases,
you'll
> have to do the research on that. Let us know if you have further
(GROMACS-related) issues.
> Cheers,
> --
> Szil?rd
> On Fri, Mar 1, 2013 at 2:48 PM, George Patargias 
wrote:
>> Hi Szilαrd
>> Thanks for your reply. I have run the deviceQuery utility and what I
got
>> back is
>> /deviceQuery Starting...
>>  CUDA Device Query (Runtime API) version (CUDART static linking)
>> cudaGetDeviceCount returned 38
>> -> no CUDA-capable device is detected
>> Should I understand from this that the CUDA driver was not installed from
>> the MAC OS  X CUDA 5.0 Production Release?
>> George
>> > HI,
>> > That looks like the driver does not work or is incompatible with the
runtime. Please get the SDK, compile a simple program, e.g.
>> deviceQuery
>> > and
>> > see if that works (I suspect that it won't).
>> > Regarding your machines, just FYI, the Quadro 4000 is a pretty slow
>> card
>> > (somewhat slower than a GTX 460) so you'll hava a quite strong
>> resource
>> > imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and
>> little
>> > GPU
>> > compute power which will lead to the CPU idling while waiting for the
>> GPU.
>> > Cheers,
>> > --
>> > Szilαrd
>> > On Thu, Feb 28, 2013 at 4:52 PM, George Patargias
>> > wrote:
>> >> Hello
>> >> We are trying to install the GPU version of GROMACS 4.6 on our own
MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro
>> 4000
>> >> cards:
>> >> - We have downloaded and install the MAC OS  X CUDA 5.0 Production
Release
>> >> from here: https://developer.nvidia.com/cuda-downloads
>> >> placing the libraries contained in this download in
>> /usr/local/cuda/lib
>> >> - We have managed to compile GROMACS 4.6 linking it statically with
these
>> >> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
GMX_PREFER_STATIC_LIBS=ON)
>> >> Unfortunately, when we tried to run a test job with the generated
mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
devices. It also reports 0.0 version for CUDA driver and runtime. Is
the actual CUDA driver missing from the MAC OS  X CUDA 5.0
>> Production
>> >> Release that we installed? Do we need to install it from here:
http://www.nvidia.com/object/cuda-mac-driver.html
>> >> Or is something else that we need to do?
>> >> Many thanks in advance.
>> >> George
>> >> Dr. George Patargias
>> >> Postdoctoral Researcher
>> >> Biomedical Research Foundation
>> >> Academy of Athens
>> >> 4, Soranou Ephessiou
>> >> 115 27
>> >> Athens
>> >> Greece
>> >> Office: +302106597568
>> >> --
>> >> gmx-users mailing listgmx-users@gromacs.org
>> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> >> * Please search the archive at
>> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the www
interface or send it to gmx-users-requ...@gromacs.org.
>> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> > --
>> > gmx-users mailing listgmx-users@gromacs.org
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting! *
Please don't post (un)subscribe requests to the list. Use the www
interface or send it to gmx-users-requ...@gromacs.org.
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> Dr. George Patargias
>> Postdoctoral Researcher
>> Biomedical Research Foundation
>> Academy of Athens
>> 4, Soranou Ephessiou
>> 115 27
>> Athens
>> Greece
>> Office: +302106597568
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting! *
Please don't post (un)subscribe requests to the list. Use the www
interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[Albert]

The easiest way for solution is to kill MacOS ans switch to Linux.

;-)

Albert


On 03/01/2013 06:03 PM, Szilárd Páll wrote:

Hi George,

As I said before, that just means that most probably the GPU driver is not
compatible with the CUDA runtime (libcudart) that you installed with the
CUDA toolkit. I've no clue about the Mac OS installers and releases, you'll
have to do the research on that. Let us know if you have further
(GROMACS-related) issues.

Cheers,

--
Szilárd


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[Szilárd Páll]

Hi George,

As I said before, that just means that most probably the GPU driver is not
compatible with the CUDA runtime (libcudart) that you installed with the
CUDA toolkit. I've no clue about the Mac OS installers and releases, you'll
have to do the research on that. Let us know if you have further
(GROMACS-related) issues.

Cheers,

--
Szilárd


On Fri, Mar 1, 2013 at 2:48 PM, George Patargias  wrote:

> Hi Szilαrd
>
> Thanks for your reply. I have run the deviceQuery utility and what I got
> back is
>
> /deviceQuery Starting...
>
>  CUDA Device Query (Runtime API) version (CUDART static linking)
>
> cudaGetDeviceCount returned 38
> -> no CUDA-capable device is detected
>
> Should I understand from this that the CUDA driver was not installed from
> the MAC OS  X CUDA 5.0 Production Release?
>
> George
>
>
> > HI,
> >
> > That looks like the driver does not work or is incompatible with the
> > runtime. Please get the SDK, compile a simple program, e.g. deviceQuery
> > and
> > see if that works (I suspect that it won't).
> >
> > Regarding your machines, just FYI, the Quadro 4000 is a pretty slow card
> > (somewhat slower than a GTX 460) so you'll hava a quite strong resource
> > imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and little
> > GPU
> > compute power which will lead to the CPU idling while waiting for the
> GPU.
> >
> > Cheers,
> >
> > --
> > Szilαrd
> >
> >
> > On Thu, Feb 28, 2013 at 4:52 PM, George Patargias
> > wrote:
> >
> >> Hello
> >>
> >> We are trying to install the GPU version of GROMACS 4.6 on our own
> >> MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
> >> cards:
> >>
> >> - We have downloaded and install the MAC OS  X CUDA 5.0 Production
> >> Release
> >> from here: https://developer.nvidia.com/cuda-downloads
> >>
> >> placing the libraries contained in this download in /usr/local/cuda/lib
> >>
> >> - We have managed to compile GROMACS 4.6 linking it statically with
> >> these
> >> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
> >> GMX_PREFER_STATIC_LIBS=ON)
> >>
> >> Unfortunately, when we tried to run a test job with the generated
> >> mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
> >> devices. It also reports 0.0 version for CUDA driver and runtime.
> >>
> >> Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
> >> Release that we installed? Do we need to install it from here:
> >>
> >> http://www.nvidia.com/object/cuda-mac-driver.html
> >>
> >> Or is something else that we need to do?
> >>
> >> Many thanks in advance.
> >> George
> >>
> >>
> >> Dr. George Patargias
> >> Postdoctoral Researcher
> >> Biomedical Research Foundation
> >> Academy of Athens
> >> 4, Soranou Ephessiou
> >> 115 27
> >> Athens
> >> Greece
> >>
> >> Office: +302106597568
> >>
> >>
> >> --
> >> gmx-users mailing listgmx-users@gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >> * Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-users-requ...@gromacs.org.
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
>
>
> Dr. George Patargias
> Postdoctoral Researcher
> Biomedical Research Foundation
> Academy of Athens
> 4, Soranou Ephessiou
> 115 27
> Athens
> Greece
>
> Office: +302106597568
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[George Patargias]

Hi Szilαrd

Thanks for your reply. I have run the deviceQuery utility and what I got
back is

/deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 38
-> no CUDA-capable device is detected

Should I understand from this that the CUDA driver was not installed from
the MAC OS  X CUDA 5.0 Production Release?

George


> HI,
>
> That looks like the driver does not work or is incompatible with the
> runtime. Please get the SDK, compile a simple program, e.g. deviceQuery
> and
> see if that works (I suspect that it won't).
>
> Regarding your machines, just FYI, the Quadro 4000 is a pretty slow card
> (somewhat slower than a GTX 460) so you'll hava a quite strong resource
> imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and little
> GPU
> compute power which will lead to the CPU idling while waiting for the GPU.
>
> Cheers,
>
> --
> Szilαrd
>
>
> On Thu, Feb 28, 2013 at 4:52 PM, George Patargias
> wrote:
>
>> Hello
>>
>> We are trying to install the GPU version of GROMACS 4.6 on our own
>> MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
>> cards:
>>
>> - We have downloaded and install the MAC OS  X CUDA 5.0 Production
>> Release
>> from here: https://developer.nvidia.com/cuda-downloads
>>
>> placing the libraries contained in this download in /usr/local/cuda/lib
>>
>> - We have managed to compile GROMACS 4.6 linking it statically with
>> these
>> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
>> GMX_PREFER_STATIC_LIBS=ON)
>>
>> Unfortunately, when we tried to run a test job with the generated
>> mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
>> devices. It also reports 0.0 version for CUDA driver and runtime.
>>
>> Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
>> Release that we installed? Do we need to install it from here:
>>
>> http://www.nvidia.com/object/cuda-mac-driver.html
>>
>> Or is something else that we need to do?
>>
>> Many thanks in advance.
>> George
>>
>>
>> Dr. George Patargias
>> Postdoctoral Researcher
>> Biomedical Research Foundation
>> Academy of Athens
>> 4, Soranou Ephessiou
>> 115 27
>> Athens
>> Greece
>>
>> Office: +302106597568
>>
>>
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>


Dr. George Patargias
Postdoctoral Researcher
Biomedical Research Foundation
Academy of Athens
4, Soranou Ephessiou
115 27
Athens
Greece

Office: +302106597568

-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[Szilárd Páll]

HI,

That looks like the driver does not work or is incompatible with the
runtime. Please get the SDK, compile a simple program, e.g. deviceQuery and
see if that works (I suspect that it won't).

Regarding your machines, just FYI, the Quadro 4000 is a pretty slow card
(somewhat slower than a GTX 460) so you'll hava a quite strong resource
imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and little GPU
compute power which will lead to the CPU idling while waiting for the GPU.

Cheers,

--
Szilárd


On Thu, Feb 28, 2013 at 4:52 PM, George Patargias wrote:

> Hello
>
> We are trying to install the GPU version of GROMACS 4.6 on our own
> MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
> cards:
>
> - We have downloaded and install the MAC OS  X CUDA 5.0 Production Release
> from here: https://developer.nvidia.com/cuda-downloads
>
> placing the libraries contained in this download in /usr/local/cuda/lib
>
> - We have managed to compile GROMACS 4.6 linking it statically with these
> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
> GMX_PREFER_STATIC_LIBS=ON)
>
> Unfortunately, when we tried to run a test job with the generated
> mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
> devices. It also reports 0.0 version for CUDA driver and runtime.
>
> Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
> Release that we installed? Do we need to install it from here:
>
> http://www.nvidia.com/object/cuda-mac-driver.html
>
> Or is something else that we need to do?
>
> Many thanks in advance.
> George
>
>
> Dr. George Patargias
> Postdoctoral Researcher
> Biomedical Research Foundation
> Academy of Athens
> 4, Soranou Ephessiou
> 115 27
> Athens
> Greece
>
> Office: +302106597568
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[George Patargias]

Hello

We are trying to install the GPU version of GROMACS 4.6 on our own
MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
cards:

- We have downloaded and install the MAC OS  X CUDA 5.0 Production Release
from here: https://developer.nvidia.com/cuda-downloads

placing the libraries contained in this download in /usr/local/cuda/lib

- We have managed to compile GROMACS 4.6 linking it statically with these
CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
GMX_PREFER_STATIC_LIBS=ON)

Unfortunately, when we tried to run a test job with the generated
mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
devices. It also reports 0.0 version for CUDA driver and runtime.

Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
Release that we installed? Do we need to install it from here:

http://www.nvidia.com/object/cuda-mac-driver.html

Or is something else that we need to do?

Many thanks in advance.
George


Dr. George Patargias
Postdoctoral Researcher
Biomedical Research Foundation
Academy of Athens
4, Soranou Ephessiou
115 27
Athens
Greece

Office: +302106597568


  
  


  
   Original Message 
  

  
Subject:

Re: gromacs installation in CyTera
  
  
Date: 
Mon, 17 Dec 2012 12:41:03 +0200
  
  
From: 
George Tsouloupas 
  
  
To: 
Paraskevi Gkeka 
  
  
CC: 
Thekla Loizou 
  

  
  
  
  Ευχαρίστησή μας!

##
module purge
module load torque


ver=mpi_d
module load FFTW/3.3.1-gompi-1.1.0-FLOAT CMake/2.8.4-gompi-1.1.0 CUDA/4.2
#module load cmake gcc fftw/3.3-gcc-openmpi


rm gromacs-$ver
mkdir gromacs-$ver
#ln -s /gpfs/scratch/GPUWorkshop/gromacs-$ver .
cd gromacs-$ver
cmake .. -DGMX_MPI=ON -DGMX_OPENMP=ON -DCMAKE_INSTALL_PREFIX=./install
make
make install
##

I am not sure I left this in a working state. Let me know whether it 
works on not..

thanks,
George.


On 12/17/2012 12:33 PM, Paraskevi Gkeka wrote:
> Giwrgo geia,
>
> katarxas sygxarhthria gia th diorganwsh tou workshop - htan pragmatika 
> poly xrhsimo gia thn poreia tou project mas. H8ela na sou zhthsw an 
> mporeis na mou steileis to script me ta commands gia to compilation 
> tou Gromacs.
>
> Euxaristw,
> Evi
>



  


  
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Albert]

On 12/17/2012 08:06 PM, Justin Lemkul wrote:
It seems to me that the system is simply crashing like any other that 
becomes unstable.  Does the simulation run at all on plain CPU?


-Justin 



Thank you very much Justin, it's really helpful. I've checked that the 
structure after minization and found that there is some problem with my 
ligand. I regenerated the ligand toplogy with acpype, and resubmit for 
mimization and NVT. Now it goes well. So probably the problems comes 
from the incorrect ligand topolgy which make the system very unstable.


best
Albert
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Justin Lemkul]

On 12/17/12 2:03 PM, Albert wrote:

well, that's one of the log files.
I've tried both

VERSION 4.6-dev-20121004-5d6c49d
VERSION 4.6-beta1
VERSION 4.6-beta2
and the latest 5.0 by git.

the problems are the same.:-(



It seems to me that the system is simply crashing like any other that becomes 
unstable.  Does the simulation run at all on plain CPU?


-Justin





On 12/17/2012 07:56 PM, Mark Abraham wrote:

On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:


>hello:
>
>  I reduced the GPU to two, and it said:
>
>Back Off! I just backed up nvt.log to ./#nvt.log.1#
>Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
>

This is a development version from October 1. Please use the mdrun version
you think you're using:-)

Mark
--




--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Albert]

well, that's one of the log files.
I've tried both

VERSION 4.6-dev-20121004-5d6c49d
VERSION 4.6-beta1
VERSION 4.6-beta2
and the latest 5.0 by git.

the problems are the same.:-(




On 12/17/2012 07:56 PM, Mark Abraham wrote:

On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:


>hello:
>
>  I reduced the GPU to two, and it said:
>
>Back Off! I just backed up nvt.log to ./#nvt.log.1#
>Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
>

This is a development version from October 1. Please use the mdrun version
you think you're using:-)

Mark
--


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

On Mon, Dec 17, 2012 at 7:56 PM, Mark Abraham wrote:

> On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:
>
> > hello:
> >
> >  I reduced the GPU to two, and it said:
> >
> > Back Off! I just backed up nvt.log to ./#nvt.log.1#
> > Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
> >
>
> This is a development version from October 1. Please use the mdrun version
> you think you're using :-)
>

Thanks Mark, good catch!

--
Szilárd



>
> Mark
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Mark Abraham]

On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:

> hello:
>
>  I reduced the GPU to two, and it said:
>
> Back Off! I just backed up nvt.log to ./#nvt.log.1#
> Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
>

This is a development version from October 1. Please use the mdrun version
you think you're using :-)

Mark
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

Hi Albert,

Thanks for the testing.

Last questions.
- What version are you using? Is it beta2 release or latest git? if it's
the former, getting the latest git might help if...
-  (do) you happen to be using GMX_GPU_ACCELERATION=None (you shouldn't!)?
A bug triggered only with this setting has been fixed recently.

If the above doesn't help, please file a bug report and attach a tpr so we
can reproduce.

Cheers,

--
Szilárd



On Mon, Dec 17, 2012 at 6:21 PM, Albert  wrote:

> On 12/17/2012 06:08 PM, Szilárd Páll wrote:
>
>> Hi,
>>
>> How about GPU emulation or CPU-only runs? Also, please try setting the
>> number of therads to 1 (-ntomp 1).
>>
>>
>> --
>> Szilárd
>>
>>
> hello:
>
> I am running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
> set (and to match closer the GPU setup with -ntomp 12), it failed with log:
>
> Back Off! I just backed up step33b.pdb to ./#step33b.pdb.2#
>
> Back Off! I just backed up step33c.pdb to ./#step33c.pdb.2#
>
> Wrote pdb files with previous and current coordinates
> [CUDANodeA:20753] *** Process received signal ***
> [CUDANodeA:20753] Signal: Segmentation fault (11)
> [CUDANodeA:20753] Signal code: Address not mapped (1)
> [CUDANodeA:20753] Failing at address: 0x106ae6a00
>
> [1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g
> nvt.log -x nvt.xtc -ntomp 12
>
>
>
>
> I also tried , number of therads to 1 (-ntomp 1), it failed with following
> messages:
>
>
> Back Off! I just backed up step33c.pdb to ./#step33c.pdb.1#
>
> Wrote pdb files with previous and current coordinates
> [CUDANodeA:20740] *** Process received signal ***
> [CUDANodeA:20740] Signal: Segmentation fault (11)
> [CUDANodeA:20740] Signal code: Address not mapped (1)
> [CUDANodeA:20740] Failing at address: 0x1f74a96ec
> [CUDANodeA:20740] [ 0] /lib64/libpthread.so.0(+**0xf2d0) [0x2b351d3022d0]
> [CUDANodeA:20740] [ 1] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x11020f)
> [0x2b351a99c20f]
> [CUDANodeA:20740] [ 2] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x111c94)
> [0x2b351a99dc94]
> [CUDANodeA:20740] [ 3] 
> /opt/gromacs-4.6/lib/libmd_**mpi.so.6(gmx_pme_do+0x1d2e)
> [0x2b351a9a1bae]
> [CUDANodeA:20740] [ 4] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_lowlevel+**0x1eef) [0x2b351a97262f]
> [CUDANodeA:20740] [ 5] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_cutsVERLET+**0x1756) [0x2b351aa04736]
> [CUDANodeA:20740] [ 6] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(do_force+0x3bf)
> [0x2b351aa0a0df]
> [CUDANodeA:20740] [ 7] mdrun_mpi(do_md+0x8133) [0x4334c3]
> [CUDANodeA:20740] [ 8] mdrun_mpi(mdrunner+0x19e9) [0x411639]
> [CUDANodeA:20740] [ 9] mdrun_mpi(main+0x17db) [0x4373db]
> [CUDANodeA:20740] [10] /lib64/libc.so.6(__libc_start_**main+0xfd)
> [0x2b351d52ebfd]
> [CUDANodeA:20740] [11] mdrun_mpi() [0x407f09]
> [CUDANodeA:20740] *** End of error message ***
>
> [1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g
> nvt.log -x nvt.xtc -ntomp 1
>
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Albert]

On 12/17/2012 06:08 PM, Szilárd Páll wrote:

Hi,

How about GPU emulation or CPU-only runs? Also, please try setting the
number of therads to 1 (-ntomp 1).


--
Szilárd



hello:

I am running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
set (and to match closer the GPU setup with -ntomp 12), it failed with log:

Back Off! I just backed up step33b.pdb to ./#step33b.pdb.2#

Back Off! I just backed up step33c.pdb to ./#step33c.pdb.2#
Wrote pdb files with previous and current coordinates
[CUDANodeA:20753] *** Process received signal ***
[CUDANodeA:20753] Signal: Segmentation fault (11)
[CUDANodeA:20753] Signal code: Address not mapped (1)
[CUDANodeA:20753] Failing at address: 0x106ae6a00

[1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g 
nvt.log -x nvt.xtc -ntomp 12




I also tried , number of therads to 1 (-ntomp 1), it failed with following 
messages:


Back Off! I just backed up step33c.pdb to ./#step33c.pdb.1#
Wrote pdb files with previous and current coordinates
[CUDANodeA:20740] *** Process received signal ***
[CUDANodeA:20740] Signal: Segmentation fault (11)
[CUDANodeA:20740] Signal code: Address not mapped (1)
[CUDANodeA:20740] Failing at address: 0x1f74a96ec
[CUDANodeA:20740] [ 0] /lib64/libpthread.so.0(+0xf2d0) [0x2b351d3022d0]
[CUDANodeA:20740] [ 1] /opt/gromacs-4.6/lib/libmd_mpi.so.6(+0x11020f) 
[0x2b351a99c20f]
[CUDANodeA:20740] [ 2] /opt/gromacs-4.6/lib/libmd_mpi.so.6(+0x111c94) 
[0x2b351a99dc94]
[CUDANodeA:20740] [ 3] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(gmx_pme_do+0x1d2e) [0x2b351a9a1bae]
[CUDANodeA:20740] [ 4] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(do_force_lowlevel+0x1eef) 
[0x2b351a97262f]
[CUDANodeA:20740] [ 5] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(do_force_cutsVERLET+0x1756) 
[0x2b351aa04736]
[CUDANodeA:20740] [ 6] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(do_force+0x3bf) [0x2b351aa0a0df]

[CUDANodeA:20740] [ 7] mdrun_mpi(do_md+0x8133) [0x4334c3]
[CUDANodeA:20740] [ 8] mdrun_mpi(mdrunner+0x19e9) [0x411639]
[CUDANodeA:20740] [ 9] mdrun_mpi(main+0x17db) [0x4373db]
[CUDANodeA:20740] [10] /lib64/libc.so.6(__libc_start_main+0xfd) 
[0x2b351d52ebfd]

[CUDANodeA:20740] [11] mdrun_mpi() [0x407f09]
[CUDANodeA:20740] *** End of error message ***

[1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro 
-g nvt.log -x nvt.xtc -ntomp 1




--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

Hi,

How about GPU emulation or CPU-only runs? Also, please try setting the
number of therads to 1 (-ntomp 1).


--
Szilárd



On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:

> hello:
>
>  I reduced the GPU to two, and it said:
>
> Back Off! I just backed up nvt.log to ./#nvt.log.1#
> Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
>
> NOTE: GPU(s) found, but the current simulation can not use GPUs
>   To use a GPU, set the mdp option: cutoff-scheme = Verlet
>   (for quick performance testing you can use the -testverlet option)
>
> Using 2 MPI processes
>
> 4 GPUs detected on host CUDANodeA:
>   #0: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #1: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #2: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>   #3: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
>
> Making 1D domain decomposition 2 x 1 x 1
>
> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> We have just committed the new CPU detection code in this branch,
> and will commit new SSE/AVX kernels in a few days. However, this
> means that currently only the NxN kernels are accelerated!
> In the mean time, you might want to avoid production runs in 4.6.
>
>
> when I run it with single GPU, it produced lots of pdb file with prefix
> "step", and then it crashed with messages:
>
> Wrote pdb files with previous and current coordinates
> Warning: 1-4 interaction between 4674 and 4706 at distance 434.986 which
> is larger than the 1-4 table size 2.200 nm
> These are ignored for the rest of the simulation
> This usually means your system is exploding,
> if not, you should increase table-extension in your mdp file
> or with user tables increase the table size
> [CUDANodeA:20659] *** Process received signal ***
> [CUDANodeA:20659] Signal: Segmentation fault (11)
> [CUDANodeA:20659] Signal code: Address not mapped (1)
> [CUDANodeA:20659] Failing at address: 0xc7aa00dc
> [CUDANodeA:20659] [ 0] /lib64/libpthread.so.0(+**0xf2d0) [0x2ab25c76d2d0]
> [CUDANodeA:20659] [ 1] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x11020f)
> [0x2ab259e0720f]
> [CUDANodeA:20659] [ 2] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x111c94)
> [0x2ab259e08c94]
> [CUDANodeA:20659] [ 3] 
> /opt/gromacs-4.6/lib/libmd_**mpi.so.6(gmx_pme_do+0x1d2e)
> [0x2ab259e0cbae]
> [CUDANodeA:20659] [ 4] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_lowlevel+**0x1eef) [0x2ab259ddd62f]
> [CUDANodeA:20659] [ 5] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_cutsGROUP+**0x1495) [0x2ab259e72a45]
> [CUDANodeA:20659] [ 6] mdrun_mpi(do_md+0x8133) [0x4334c3]
> [CUDANodeA:20659] [ 7] mdrun_mpi(mdrunner+0x19e9) [0x411639]
> [CUDANodeA:20659] [ 8] mdrun_mpi(main+0x17db) [0x4373db]
> [CUDANodeA:20659] [ 9] /lib64/libc.so.6(__libc_start_**main+0xfd)
> [0x2ab25c999bfd]
> [CUDANodeA:20659] [10] mdrun_mpi() [0x407f09]
> [CUDANodeA:20659] *** End of error message ***
>
> [1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g
> nvt.log -x nvt.xtc
>
>
>
> here is the .mdp file I used:
>
> title   = NVT equilibration for OR-POPC system
> define  = -DPOSRES -DPOSRES_LIG ; Protein is position restrained
> (uses the posres.itp file information)
> ; Parameters describing the details of the NVT simulation protocol
> integrator  = md; Algorithm ("md" = molecular dynamics
> [leap-frog integrator]; "md-vv" = md using velocity verlet; sd = stochastic
> dynamics)
> dt  = 0.002 ; Time-step (ps)
> nsteps  = 25; Number of steps to run (0.002 * 25 =
> 500 ps)
>
> ; Parameters controlling output writing
> nstxout = 0 ; Write coordinates to output .trr file
> every 2 ps
> nstvout = 0 ; Write velocities to output .trr file
> every 2 ps
> nstfout = 0
>
> nstxtcout   = 1000
> nstenergy   = 1000  ; Write energies to output .edr file every
> 2 ps
> nstlog  = 1000  ; Write output to .log file every 2 ps
>
> ; Parameters describing neighbors searching and details about interaction
> calculations
> ns_type = grid  ; Neighbor list search method (simple,
> grid)
> nstlist = 50; Neighbor list update frequency (after
> every given number of steps)
> rlist   = 1.2   ; Neighbor list search cut-off distance
> (nm)
> rlistlong   = 1.4
> rcoulomb= 1.2   ; Short-range Coulombic interactions
> cut-off distance (nm)
> rvdw= 1.2   ; Short-range van der Waals cutoff
> distance (nm)
> pbc = xyz   ; Direction in which to use Perodic
> Boundary Conditions (xyz, xy, no)
> cutoff-scheme   =Verlet  ; GPU running
>
> ; Parameters for treating bonded interactions
> continuation= no; Whether a fresh start or a continuation
> from a previous run (yes/no)
> constraint_algorithm = LINCS

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Albert]

hello:

 I reduced the GPU to two, and it said:

Back Off! I just backed up nvt.log to ./#nvt.log.1#
Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)

NOTE: GPU(s) found, but the current simulation can not use GPUs
  To use a GPU, set the mdp option: cutoff-scheme = Verlet
  (for quick performance testing you can use the -testverlet option)

Using 2 MPI processes

4 GPUs detected on host CUDANodeA:
  #0: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible
  #3: NVIDIA GeForce GTX 590, compute cap.: 2.0, ECC:  no, stat: compatible

Making 1D domain decomposition 2 x 1 x 1

* WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
We have just committed the new CPU detection code in this branch,
and will commit new SSE/AVX kernels in a few days. However, this
means that currently only the NxN kernels are accelerated!
In the mean time, you might want to avoid production runs in 4.6.


when I run it with single GPU, it produced lots of pdb file with prefix 
"step", and then it crashed with messages:


Wrote pdb files with previous and current coordinates
Warning: 1-4 interaction between 4674 and 4706 at distance 434.986 which 
is larger than the 1-4 table size 2.200 nm

These are ignored for the rest of the simulation
This usually means your system is exploding,
if not, you should increase table-extension in your mdp file
or with user tables increase the table size
[CUDANodeA:20659] *** Process received signal ***
[CUDANodeA:20659] Signal: Segmentation fault (11)
[CUDANodeA:20659] Signal code: Address not mapped (1)
[CUDANodeA:20659] Failing at address: 0xc7aa00dc
[CUDANodeA:20659] [ 0] /lib64/libpthread.so.0(+0xf2d0) [0x2ab25c76d2d0]
[CUDANodeA:20659] [ 1] /opt/gromacs-4.6/lib/libmd_mpi.so.6(+0x11020f) 
[0x2ab259e0720f]
[CUDANodeA:20659] [ 2] /opt/gromacs-4.6/lib/libmd_mpi.so.6(+0x111c94) 
[0x2ab259e08c94]
[CUDANodeA:20659] [ 3] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(gmx_pme_do+0x1d2e) [0x2ab259e0cbae]
[CUDANodeA:20659] [ 4] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(do_force_lowlevel+0x1eef) 
[0x2ab259ddd62f]
[CUDANodeA:20659] [ 5] 
/opt/gromacs-4.6/lib/libmd_mpi.so.6(do_force_cutsGROUP+0x1495) 
[0x2ab259e72a45]

[CUDANodeA:20659] [ 6] mdrun_mpi(do_md+0x8133) [0x4334c3]
[CUDANodeA:20659] [ 7] mdrun_mpi(mdrunner+0x19e9) [0x411639]
[CUDANodeA:20659] [ 8] mdrun_mpi(main+0x17db) [0x4373db]
[CUDANodeA:20659] [ 9] /lib64/libc.so.6(__libc_start_main+0xfd) 
[0x2ab25c999bfd]

[CUDANodeA:20659] [10] mdrun_mpi() [0x407f09]
[CUDANodeA:20659] *** End of error message ***

[1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro 
-g nvt.log -x nvt.xtc




here is the .mdp file I used:

title   = NVT equilibration for OR-POPC system
define  = -DPOSRES -DPOSRES_LIG ; Protein is position restrained 
(uses the posres.itp file information)

; Parameters describing the details of the NVT simulation protocol
integrator  = md; Algorithm ("md" = molecular dynamics 
[leap-frog integrator]; "md-vv" = md using velocity verlet; sd = 
stochastic dynamics)

dt  = 0.002 ; Time-step (ps)
nsteps  = 25; Number of steps to run (0.002 * 25 
= 500 ps)


; Parameters controlling output writing
nstxout = 0 ; Write coordinates to output .trr file 
every 2 ps
nstvout = 0 ; Write velocities to output .trr file 
every 2 ps

nstfout = 0

nstxtcout   = 1000
nstenergy   = 1000  ; Write energies to output .edr file 
every 2 ps

nstlog  = 1000  ; Write output to .log file every 2 ps

; Parameters describing neighbors searching and details about 
interaction calculations

ns_type = grid  ; Neighbor list search method (simple, grid)
nstlist = 50; Neighbor list update frequency (after 
every given number of steps)

rlist   = 1.2   ; Neighbor list search cut-off distance (nm)
rlistlong   = 1.4
rcoulomb= 1.2   ; Short-range Coulombic interactions 
cut-off distance (nm)
rvdw= 1.2   ; Short-range van der Waals cutoff 
distance (nm)
pbc = xyz   ; Direction in which to use Perodic 
Boundary Conditions (xyz, xy, no)

cutoff-scheme   =Verlet  ; GPU running

; Parameters for treating bonded interactions
continuation= no; Whether a fresh start or a 
continuation from a previous run (yes/no)

constraint_algorithm = LINCS; Constraint algorithm (LINCS / SHAKE)
constraints = all-bonds ; Which bonds/angles to constrain 
(all-bonds / hbonds / none / all-angles / h-angles)
lincs_iter  = 1 ; Number of iterations to correct for 
rotational lengthening in LINCS (related to accuracy)
lincs_order = 4 ; Highest order in the expansion of the 
const

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

Hi,

That unfortunately tell exactly about the reason why mdrun is stuck. Can
you reproduce the issue on another machines or with different launch
configurations? At which step does it get stuck (-stepout 1 can help)?

Please try the following:
- try running on a single GPU;
- try running on CPUs only (-nb cpu and to match closer the GPU setup with
-ntomp 12);
- try running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
set (and to match closer the GPU setup with -ntomp 12)
- provide a backtrace (using gdb).

Cheers,

--
Szilárd



On Mon, Dec 17, 2012 at 5:37 PM, Albert  wrote:

> hello:
>
>  I am running GMX-4.6 beta2 GPU work in a 24 CPU core workstation with two
> GTX590, it stacked there without any output i.e the .xtc file size is
> always 0 after hours of running. Here is the md.log file I found:
>
>
> Using CUDA 8x8x8 non-bonded kernels
>
> Potential shift: LJ r^-12: 0.112 r^-6 0.335, Ewald 1.000e-05
> Initialized non-bonded Ewald correction tables, spacing: 7.82e-04 size:
> 1536
>
> Removing pbc first time
> Pinning to Hyper-Threading cores with 12 physical cores in a compute node
> There are 1 flexible constraints
>
> WARNING: step size for flexible constraining = 0
>  All flexible constraints will be rigid.
>  Will try to keep all flexible constraints at their original
> length,
>  but the lengths may exhibit some drift.
>
> Initializing Parallel LINear Constraint Solver
> Linking all bonded interactions to atoms
> There are 161872 inter charge-group exclusions,
> will use an extra communication step for exclusion forces for PME
>
> The initial number of communication pulses is: X 1
> The initial domain decomposition cell size is: X 1.83 nm
>
> The maximum allowed distance for charge groups involved in interactions is:
>  non-bonded interactions   1.200 nm
> (the following are initial values, they could change due to box
> deformation)
> two-body bonded interactions  (-rdd)   1.200 nm
>   multi-body bonded interactions  (-rdd)   1.200 nm
>   atoms separated by up to 5 constraints  (-rcon)  1.826 nm
>
> When dynamic load balancing gets turned on, these settings will change to:
> The maximum number of communication pulses is: X 1
> The minimum size for domain decomposition cells is 1.200 nm
> The requested allowed shrink of DD cells (option -dds) is: 0.80
> The allowed shrink of domain decomposition cells is: X 0.66
> The maximum allowed distance for charge groups involved in interactions is:
>  non-bonded interactions   1.200 nm
> two-body bonded interactions  (-rdd)   1.200 nm
>   multi-body bonded interactions  (-rdd)   1.200 nm
>   atoms separated by up to 5 constraints  (-rcon)  1.200 nm
>
> Making 1D domain decomposition grid 4 x 1 x 1, home cell index 0 0 0
>
> Center of mass motion removal mode is Linear
> We have the following groups for center of mass motion removal:
>   0:  Protein_LIG_POPC
>   1:  Water_and_ions
>
>  PLEASE READ AND CITE THE FOLLOWING REFERENCE 
> G. Bussi, D. Donadio and M. Parrinello
> Canonical sampling through velocity rescaling
> J. Chem. Phys. 126 (2007) pp. 014101
>   --- Thank You ---  
>
>
>
> THX
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU running problem with GMX-4.6 beta2

[Albert]

hello:

 I am running GMX-4.6 beta2 GPU work in a 24 CPU core workstation with 
two GTX590, it stacked there without any output i.e the .xtc file size 
is always 0 after hours of running. Here is the md.log file I found:



Using CUDA 8x8x8 non-bonded kernels

Potential shift: LJ r^-12: 0.112 r^-6 0.335, Ewald 1.000e-05
Initialized non-bonded Ewald correction tables, spacing: 7.82e-04 size: 1536

Removing pbc first time
Pinning to Hyper-Threading cores with 12 physical cores in a compute node
There are 1 flexible constraints

WARNING: step size for flexible constraining = 0
 All flexible constraints will be rigid.
 Will try to keep all flexible constraints at their original 
length,

 but the lengths may exhibit some drift.

Initializing Parallel LINear Constraint Solver
Linking all bonded interactions to atoms
There are 161872 inter charge-group exclusions,
will use an extra communication step for exclusion forces for PME

The initial number of communication pulses is: X 1
The initial domain decomposition cell size is: X 1.83 nm

The maximum allowed distance for charge groups involved in interactions is:
 non-bonded interactions   1.200 nm
(the following are initial values, they could change due to box deformation)
two-body bonded interactions  (-rdd)   1.200 nm
  multi-body bonded interactions  (-rdd)   1.200 nm
  atoms separated by up to 5 constraints  (-rcon)  1.826 nm

When dynamic load balancing gets turned on, these settings will change to:
The maximum number of communication pulses is: X 1
The minimum size for domain decomposition cells is 1.200 nm
The requested allowed shrink of DD cells (option -dds) is: 0.80
The allowed shrink of domain decomposition cells is: X 0.66
The maximum allowed distance for charge groups involved in interactions is:
 non-bonded interactions   1.200 nm
two-body bonded interactions  (-rdd)   1.200 nm
  multi-body bonded interactions  (-rdd)   1.200 nm
  atoms separated by up to 5 constraints  (-rcon)  1.200 nm

Making 1D domain decomposition grid 4 x 1 x 1, home cell index 0 0 0

Center of mass motion removal mode is Linear
We have the following groups for center of mass motion removal:
  0:  Protein_LIG_POPC
  1:  Water_and_ions

 PLEASE READ AND CITE THE FOLLOWING REFERENCE 
G. Bussi, D. Donadio and M. Parrinello
Canonical sampling through velocity rescaling
J. Chem. Phys. 126 (2007) pp. 014101
  --- Thank You ---  



THX
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

On Tue, Dec 11, 2012 at 6:49 PM, Mirco Wahab <
mirco.wa...@chemie.tu-freiberg.de> wrote:

> Am 11.12.2012 16:04, schrieb Szilárd Páll:
>
>  It looks like some gcc 4.7-s don't work with CUDA, although I've been
>> using
>> various Ubuntu/Linaro versions, most recently 4.7.2 and had no
>> issues whatsoever. Some people seem to have bumped into the same problem
>> (see http://goo.gl/1onBz or http://goo.gl/JEnuk) and the suggested fix is
>> to put
>> #undef _GLIBCXX_ATOMIC_BUILTINS
>> #undef _GLIBCXX_USE_INT128
>> in a header and pre-include it for nvcc by calling it like this:
>> nvcc --pre-include undef_atomics_int128.h
>>
>
> The same problem occurs in SuSE 12.2/x64 with it's default 4.7.2
> (20120920).
>
> Another possible fix on SuSE 12.2: install the (older) gcc repository
> from 12.1/x64 (with lower priority), install the gcc/g++ 4.6 from there
> as an alternative compiler and select the "active gcc" through the
> "update-alternatives --config gcc" mechanism. This works very well.
>

Thanks for the info. The Ubuntu/Linaro version must have a fix for
this. Unfortunately, we can't do much about it and gcc 4.7 is anyway
blocked by the CUDA 5.0 headers.

FYI: Verlet scheme nonbonded kernels (and probably the group scheme as
well), especially with AVX, can be quite a bit slower with older gcc
versions.

I find it really annoying (and stupid) that NVIDIA did not fix their
compiler to work with gcc 4.7 which had already been out for almost a half
a year at the time of the CUDA 5.0 release.

--
Szilárd


>
> Regards
>
> M.
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Mirco Wahab]

Am 11.12.2012 16:04, schrieb Szilárd Páll:

It looks like some gcc 4.7-s don't work with CUDA, although I've been using
various Ubuntu/Linaro versions, most recently 4.7.2 and had no
issues whatsoever. Some people seem to have bumped into the same problem
(see http://goo.gl/1onBz or http://goo.gl/JEnuk) and the suggested fix is
to put
#undef _GLIBCXX_ATOMIC_BUILTINS
#undef _GLIBCXX_USE_INT128
in a header and pre-include it for nvcc by calling it like this:
nvcc --pre-include undef_atomics_int128.h


The same problem occurs in SuSE 12.2/x64 with it's default 4.7.2
(20120920).

Another possible fix on SuSE 12.2: install the (older) gcc repository
from 12.1/x64 (with lower priority), install the gcc/g++ 4.6 from there
as an alternative compiler and select the "active gcc" through the
"update-alternatives --config gcc" mechanism. This works very well.

Regards

M.

--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi Thomas,

It looks like some gcc 4.7-s don't work with CUDA, although I've been using
various Ubuntu/Linaro versions, most recently 4.7.2 and had no
issues whatsoever. Some people seem to have bumped into the same problem
(see http://goo.gl/1onBz or http://goo.gl/JEnuk) and the suggested fix is
to put
#undef _GLIBCXX_ATOMIC_BUILTINS
#undef _GLIBCXX_USE_INT128
in a header and pre-include it for nvcc by calling it like this:
nvcc --pre-include undef_atomics_int128.h

Cheers,

--
Szilárd



On Sun, Dec 9, 2012 at 12:18 PM, Thomas Evangelidis wrote:

> > > gcc 4.7.2 is not supported by any CUDA version.
> > >
> >
> > I suggest that you just fix it by editing the include/host_config.h and
> > changing the version check macro (line 82 AFAIK). I've never had real
> > problems with using new and officially not supported gcc-s, the version
> > check is more of a promise from NVIDIA that "we've tested thoroughly
> > internally and we more or less vouch for thins combination".
> >
> > Cheers,
> > --
> > Szilárd
> >
> > PS:
> > Disclamer: I don't take responsibility if your machine goes up in flames!
> > ;)
> >
> >
> Hi Szilárd,,
>
> I tried to compile gromacs-4.6beta1, is this the version you suggested? If
> not, please indicate how to download the source cause I am confused with
> all these development versions.
>
> Anyway, this is the error I get with 4.6beta1, gcc 4.7.2 and cuda 5:
>
> [  0%] Building NVCC (Device) object
>
> src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_cudautils.cu.o
>
> /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/ext/atomicity.h(48):
> error: identifier "__atomic_fetch_add" is undefined
>
>
> /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/ext/atomicity.h(52):
> error: identifier "__atomic_fetch_add" is undefined
>
> 2 errors detected in the compilation of
> "/tmp/tmpxft_2394_-9_cudautils.compute_30.cpp1.ii".
> CMake Error at cuda_tools_generated_cudautils.cu.o.cmake:252 (message):
>   Error generating file
>
>
> /home/thomas/Programs/gromacs-4.6-beta1_gnu_cuda5_build/src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_cudautils.cu.o
>
>
> gmake[3]: ***
>
> [src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir/./cuda_tools_generated_cudautils.cu.o]
> Error 1
> gmake[2]: *** [src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir/all]
> Error 2
> gmake[1]: *** [src/programs/mdrun/CMakeFiles/mdrun.dir/rule] Error 2
> gmake: *** [mdrun] Error 2
>
>
> Unless I am missing something, cuda 5 does not support gcc 4.7.2.
>
>
>  Thomas
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU compatibility

[Mark Abraham]

Correct, C1060 does not have the CUDA 2.0 compute capability required for
GROMACS 4.6. We will not have the ability to support GPU cards of lower
capability in the future. Unfortunately, your only GROMACS options are
probably to use the OpenMM functionality in 4.5.x (which is still present
in 4.6, works as far as we know, but is not in our regular test suite and
the feature is probably headed for deprecation). This will not perform as
well as the new native GPU acceleration, and supports a smaller range of
features, but might be better than wasting the GPUs.

Regards,

Mark

On Mon, Dec 10, 2012 at 7:50 AM, Cara Kreck  wrote:

>
>
>
>
> Hi,
>
> We've got a GPU cluster in our group and have really been looking forward
> to running gromacs on it with full functionality. Unfortunately, it looks
> like our NVIDIA Tesla C1060 cards aren't supported by the 4.6 beta. I was
> just wondering if there was any chance that they would be supported in the
> full version? These cards are only a couple of years old now and were
> bought specifically for running MD.
>
> Thanks,
>
> Cara
>
>   --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] GPU compatibility

[Cara Kreck]

Hi,

We've got a GPU cluster in our group and have really been looking forward to 
running gromacs on it with full functionality. Unfortunately, it looks like our 
NVIDIA Tesla C1060 cards aren't supported by the 4.6 beta. I was just wondering 
if there was any chance that they would be supported in the full version? These 
cards are only a couple of years old now and were bought specifically for 
running MD.

Thanks,

Cara

  --
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

> > gcc 4.7.2 is not supported by any CUDA version.
> >
>
> I suggest that you just fix it by editing the include/host_config.h and
> changing the version check macro (line 82 AFAIK). I've never had real
> problems with using new and officially not supported gcc-s, the version
> check is more of a promise from NVIDIA that "we've tested thoroughly
> internally and we more or less vouch for thins combination".
>
> Cheers,
> --
> Szilárd
>
> PS:
> Disclamer: I don't take responsibility if your machine goes up in flames!
> ;)
>
>
Hi Szilárd,,

I tried to compile gromacs-4.6beta1, is this the version you suggested? If
not, please indicate how to download the source cause I am confused with
all these development versions.

Anyway, this is the error I get with 4.6beta1, gcc 4.7.2 and cuda 5:

[  0%] Building NVCC (Device) object
src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_cudautils.cu.o
/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/ext/atomicity.h(48):
error: identifier "__atomic_fetch_add" is undefined

/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/ext/atomicity.h(52):
error: identifier "__atomic_fetch_add" is undefined

2 errors detected in the compilation of
"/tmp/tmpxft_2394_-9_cudautils.compute_30.cpp1.ii".
CMake Error at cuda_tools_generated_cudautils.cu.o.cmake:252 (message):
  Error generating file

/home/thomas/Programs/gromacs-4.6-beta1_gnu_cuda5_build/src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_cudautils.cu.o


gmake[3]: ***
[src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir/./cuda_tools_generated_cudautils.cu.o]
Error 1
gmake[2]: *** [src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir/all]
Error 2
gmake[1]: *** [src/programs/mdrun/CMakeFiles/mdrun.dir/rule] Error 2
gmake: *** [mdrun] Error 2


Unless I am missing something, cuda 5 does not support gcc 4.7.2.


 Thomas
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

On Sun, Nov 25, 2012 at 8:47 PM, Thomas Evangelidis wrote:

> Hi Szilárd,
>
> I was able to run code compiled with icc 13 on Fedora 17, but as I don't
> > have Intel Compiler v13 on this machine I can't check it now.
> >
> > Please check if it works for you with gcc 4.7.2 (which is the default)
> and
> > let me know if you succeed. The performance difference between icc and
> gcc
> > on your processor should be negligible with GPU runs and at most 5-10%
> with
> > CPU-only runs.
> >
> > As the issue is quite annoying, I'll try to have a look later, probably
> > after the beta is out.
> >
> >
> gcc 4.7.2 is not supported by any CUDA version.
>

I suggest that you just fix it by editing the include/host_config.h and
changing the version check macro (line 82 AFAIK). I've never had real
problems with using new and officially not supported gcc-s, the version
check is more of a promise from NVIDIA that "we've tested thoroughly
internally and we more or less vouch for thins combination".

Cheers,
--
Szilárd

PS:
Disclamer: I don't take responsibility if your machine goes up in flames! ;)


>
> Thomas
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

Hi Szilárd,

I was able to run code compiled with icc 13 on Fedora 17, but as I don't
> have Intel Compiler v13 on this machine I can't check it now.
>
> Please check if it works for you with gcc 4.7.2 (which is the default) and
> let me know if you succeed. The performance difference between icc and gcc
> on your processor should be negligible with GPU runs and at most 5-10% with
> CPU-only runs.
>
> As the issue is quite annoying, I'll try to have a look later, probably
> after the beta is out.
>
>
gcc 4.7.2 is not supported by any CUDA version.

Thomas
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

On Mon, Nov 19, 2012 at 6:25 PM, Szilárd Páll wrote:

> On Mon, Nov 19, 2012 at 4:09 PM, Thomas Evangelidis wrote:
>
>> Hi Szilárd,
>>
>> I compiled with the Intel compilers, not gcc. In case I am missing
>> something, these are the versions I have:
>>
>
> Indeed, I see it now in the log file. Let me try with icc 13 and will get
> back to you.
>

I was able to run code compiled with icc 13 on Fedora 17, but as I don't
have Intel Compiler v13 on this machine I can't check it now.

Please check if it works for you with gcc 4.7.2 (which is the default) and
let me know if you succeed. The performance difference between icc and gcc
on your processor should be negligible with GPU runs and at most 5-10% with
CPU-only runs.

As the issue is quite annoying, I'll try to have a look later, probably
after the beta is out.

Cheers,
Sz.


>
>> glibc.i6862.15-57.fc17
>> @updates
>> glibc.x86_64  2.15-57.fc17
>> @updates
>> glibc-common.x86_64   2.15-57.fc17
>> @updates
>> glibc-devel.i686  2.15-57.fc17
>> @updates
>> glibc-devel.x86_642.15-57.fc17
>> @updates
>> glibc-headers.x86_64  2.15-57.fc17   @updates
>>
>> gcc.x86_644.7.2-2.fc17
>> @updates
>> gcc-c++.x86_644.7.2-2.fc17
>> @updates
>> gcc-gfortran.x86_64   4.7.2-2.fc17
>> @updates
>> libgcc.i686   4.7.2-2.fc17
>> @updates
>> libgcc.x86_64 4.7.2-2.fc17   @updates
>>
>>
>> Thomas
>>
>>
>>
>> On 19 November 2012 16:57, Szilárd Páll  wrote:
>>
>> > Thomas & Albert,
>> >
>> > We are unable to reproduce the issue on FC 17 with glibc 2.15-58 and gcc
>> > 4.7.2.
>> >
>> > Please try to update your packages (you should have updates available
>> for
>> > glibc), try recompiling with the latest 4.6 code and report back whether
>> > you succeed.
>> >
>> > Cheers,
>> >
>> > --
>> > Szilárd
>> >
>> >
>> > On Fri, Nov 16, 2012 at 4:31 PM, Szilárd Páll > > >wrote:
>> >
>> > > Hi Albert,
>> > >
>> > > Apologies for hijacking your thread. Do you happen to have Fedora 17
>> as
>> > > well?
>> > >
>> > > --
>> > > Szilárd
>> > >
>> > >
>> > >
>> > > On Sun, Nov 4, 2012 at 10:55 AM, Albert  wrote:
>> > >
>> > >> hello:
>> > >>
>> > >>  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti
>> (2 x
>> > >> 1344 CUDA cores), and I got the following warnings:
>> > >>
>> > >> thank you very much.
>> > >>
>> > >> ---**messages--**
>> > >> -
>> > >>
>> > >> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
>> > per
>> > >> node with 2 MPI processes.
>> > >>  This will cause considerable performance loss!
>> > >>
>> > >> 2 GPUs detected on host boreas:
>> > >>   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> > >> compatible
>> > >>   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> > >> compatible
>> > >>
>> > >> 2 GPUs auto-selected to be used for this run: #0, #1
>> > >>
>> > >> Using CUDA 8x8x8 non-bonded kernels
>> > >> Making 1D domain decomposition 1 x 2 x 1
>> > >>
>> > >> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
>> > >> We have just committed the new CPU detection code in this branch,
>> > >> and will commit new SSE/AVX kernels in a few days. However, this
>> > >> means that currently only the NxN kernels are accelerated!
>> > >> In the mean time, you might want to avoid production runs in 4.6.
>> > >>
>> > >> --
>> > >> gmx-users mailing listgmx-users@gromacs.org
>> > >> http://lists.gromacs.org/**mailman/listinfo/gmx-users<
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users>
>> > >> * Please search the archive at http://www.gromacs.org/**
>> > >> Support/Mailing_Lists/Search<
>> > http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
>> > >> * Please don't post (un)subscribe requests to the list. Use the www
>> > >> interface or send it to gmx-users-requ...@gromacs.org.
>> > >> * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
>> > http://www.gromacs.org/Support/Mailing_Lists>
>> > >>
>> > >
>> > >
>> > --
>> > gmx-users mailing listgmx-users@gromacs.org
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > * Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > * Please don't post (un)subscribe requests to the list. Use the
>> > www interface or send it to gmx-users-requ...@gromacs.org.
>> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> >
>>
>>
>>
>> --
>>
>> ==
>>
>> Thomas Evangelidis
>>
>> PhD student
>> University of Athens
>> Faculty of Pharmacy
>> Department of Pharmaceutical Chemistry
>> Panepistimioupoli-Zografou
>> 157 71 Athens
>> GREECE
>>
>> email: tev...@pharm.uo

Re: [gmx-users] GPU warnings

[Szilárd Páll]

On Mon, Nov 19, 2012 at 4:09 PM, Thomas Evangelidis wrote:

> Hi Szilárd,
>
> I compiled with the Intel compilers, not gcc. In case I am missing
> something, these are the versions I have:
>

Indeed, I see it now in the log file. Let me try with icc 13 and will get
back to you.

>
> glibc.i6862.15-57.fc17
> @updates
> glibc.x86_64  2.15-57.fc17
> @updates
> glibc-common.x86_64   2.15-57.fc17
> @updates
> glibc-devel.i686  2.15-57.fc17
> @updates
> glibc-devel.x86_642.15-57.fc17
> @updates
> glibc-headers.x86_64  2.15-57.fc17   @updates
>
> gcc.x86_644.7.2-2.fc17
> @updates
> gcc-c++.x86_644.7.2-2.fc17
> @updates
> gcc-gfortran.x86_64   4.7.2-2.fc17
> @updates
> libgcc.i686   4.7.2-2.fc17
> @updates
> libgcc.x86_64 4.7.2-2.fc17   @updates
>
>
> Thomas
>
>
>
> On 19 November 2012 16:57, Szilárd Páll  wrote:
>
> > Thomas & Albert,
> >
> > We are unable to reproduce the issue on FC 17 with glibc 2.15-58 and gcc
> > 4.7.2.
> >
> > Please try to update your packages (you should have updates available for
> > glibc), try recompiling with the latest 4.6 code and report back whether
> > you succeed.
> >
> > Cheers,
> >
> > --
> > Szilárd
> >
> >
> > On Fri, Nov 16, 2012 at 4:31 PM, Szilárd Páll  > >wrote:
> >
> > > Hi Albert,
> > >
> > > Apologies for hijacking your thread. Do you happen to have Fedora 17 as
> > > well?
> > >
> > > --
> > > Szilárd
> > >
> > >
> > >
> > > On Sun, Nov 4, 2012 at 10:55 AM, Albert  wrote:
> > >
> > >> hello:
> > >>
> > >>  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2
> x
> > >> 1344 CUDA cores), and I got the following warnings:
> > >>
> > >> thank you very much.
> > >>
> > >> ---**messages--**
> > >> -
> > >>
> > >> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
> > per
> > >> node with 2 MPI processes.
> > >>  This will cause considerable performance loss!
> > >>
> > >> 2 GPUs detected on host boreas:
> > >>   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> > >> compatible
> > >>   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> > >> compatible
> > >>
> > >> 2 GPUs auto-selected to be used for this run: #0, #1
> > >>
> > >> Using CUDA 8x8x8 non-bonded kernels
> > >> Making 1D domain decomposition 1 x 2 x 1
> > >>
> > >> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> > >> We have just committed the new CPU detection code in this branch,
> > >> and will commit new SSE/AVX kernels in a few days. However, this
> > >> means that currently only the NxN kernels are accelerated!
> > >> In the mean time, you might want to avoid production runs in 4.6.
> > >>
> > >> --
> > >> gmx-users mailing listgmx-users@gromacs.org
> > >> http://lists.gromacs.org/**mailman/listinfo/gmx-users<
> > http://lists.gromacs.org/mailman/listinfo/gmx-users>
> > >> * Please search the archive at http://www.gromacs.org/**
> > >> Support/Mailing_Lists/Search<
> > http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
> > >> * Please don't post (un)subscribe requests to the list. Use the www
> > >> interface or send it to gmx-users-requ...@gromacs.org.
> > >> * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
> > http://www.gromacs.org/Support/Mailing_Lists>
> > >>
> > >
> > >
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
>
>
>
> --
>
> ==
>
> Thomas Evangelidis
>
> PhD student
> University of Athens
> Faculty of Pharmacy
> Department of Pharmaceutical Chemistry
> Panepistimioupoli-Zografou
> 157 71 Athens
> GREECE
>
> email: tev...@pharm.uoa.gr
>
>   teva...@gmail.com
>
>
> website: https://sites.google.com/site/thomasevangelidishomepage/
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

Hi Szilárd,

I compiled with the Intel compilers, not gcc. In case I am missing
something, these are the versions I have:

glibc.i6862.15-57.fc17
@updates
glibc.x86_64  2.15-57.fc17
@updates
glibc-common.x86_64   2.15-57.fc17
@updates
glibc-devel.i686  2.15-57.fc17
@updates
glibc-devel.x86_642.15-57.fc17
@updates
glibc-headers.x86_64  2.15-57.fc17   @updates

gcc.x86_644.7.2-2.fc17
@updates
gcc-c++.x86_644.7.2-2.fc17
@updates
gcc-gfortran.x86_64   4.7.2-2.fc17
@updates
libgcc.i686   4.7.2-2.fc17
@updates
libgcc.x86_64 4.7.2-2.fc17   @updates


Thomas



On 19 November 2012 16:57, Szilárd Páll  wrote:

> Thomas & Albert,
>
> We are unable to reproduce the issue on FC 17 with glibc 2.15-58 and gcc
> 4.7.2.
>
> Please try to update your packages (you should have updates available for
> glibc), try recompiling with the latest 4.6 code and report back whether
> you succeed.
>
> Cheers,
>
> --
> Szilárd
>
>
> On Fri, Nov 16, 2012 at 4:31 PM, Szilárd Páll  >wrote:
>
> > Hi Albert,
> >
> > Apologies for hijacking your thread. Do you happen to have Fedora 17 as
> > well?
> >
> > --
> > Szilárd
> >
> >
> >
> > On Sun, Nov 4, 2012 at 10:55 AM, Albert  wrote:
> >
> >> hello:
> >>
> >>  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x
> >> 1344 CUDA cores), and I got the following warnings:
> >>
> >> thank you very much.
> >>
> >> ---**messages--**
> >> -
> >>
> >> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
> per
> >> node with 2 MPI processes.
> >>  This will cause considerable performance loss!
> >>
> >> 2 GPUs detected on host boreas:
> >>   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> >> compatible
> >>   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> >> compatible
> >>
> >> 2 GPUs auto-selected to be used for this run: #0, #1
> >>
> >> Using CUDA 8x8x8 non-bonded kernels
> >> Making 1D domain decomposition 1 x 2 x 1
> >>
> >> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> >> We have just committed the new CPU detection code in this branch,
> >> and will commit new SSE/AVX kernels in a few days. However, this
> >> means that currently only the NxN kernels are accelerated!
> >> In the mean time, you might want to avoid production runs in 4.6.
> >>
> >> --
> >> gmx-users mailing listgmx-users@gromacs.org
> >> http://lists.gromacs.org/**mailman/listinfo/gmx-users<
> http://lists.gromacs.org/mailman/listinfo/gmx-users>
> >> * Please search the archive at http://www.gromacs.org/**
> >> Support/Mailing_Lists/Search<
> http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
> >> * Please don't post (un)subscribe requests to the list. Use the www
> >> interface or send it to gmx-users-requ...@gromacs.org.
> >> * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
> http://www.gromacs.org/Support/Mailing_Lists>
> >>
> >
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>



-- 

==

Thomas Evangelidis

PhD student
University of Athens
Faculty of Pharmacy
Department of Pharmaceutical Chemistry
Panepistimioupoli-Zografou
157 71 Athens
GREECE

email: tev...@pharm.uoa.gr

  teva...@gmail.com


website: https://sites.google.com/site/thomasevangelidishomepage/
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Thomas & Albert,

We are unable to reproduce the issue on FC 17 with glibc 2.15-58 and gcc
4.7.2.

Please try to update your packages (you should have updates available for
glibc), try recompiling with the latest 4.6 code and report back whether
you succeed.

Cheers,

--
Szilárd


On Fri, Nov 16, 2012 at 4:31 PM, Szilárd Páll wrote:

> Hi Albert,
>
> Apologies for hijacking your thread. Do you happen to have Fedora 17 as
> well?
>
> --
> Szilárd
>
>
>
> On Sun, Nov 4, 2012 at 10:55 AM, Albert  wrote:
>
>> hello:
>>
>>  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x
>> 1344 CUDA cores), and I got the following warnings:
>>
>> thank you very much.
>>
>> ---**messages--**
>> -
>>
>> WARNING: On node 0: oversubscribing the available 0 logical CPU cores per
>> node with 2 MPI processes.
>>  This will cause considerable performance loss!
>>
>> 2 GPUs detected on host boreas:
>>   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> compatible
>>   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> compatible
>>
>> 2 GPUs auto-selected to be used for this run: #0, #1
>>
>> Using CUDA 8x8x8 non-bonded kernels
>> Making 1D domain decomposition 1 x 2 x 1
>>
>> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
>> We have just committed the new CPU detection code in this branch,
>> and will commit new SSE/AVX kernels in a few days. However, this
>> means that currently only the NxN kernels are accelerated!
>> In the mean time, you might want to avoid production runs in 4.6.
>>
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users
>> * Please search the archive at http://www.gromacs.org/**
>> Support/Mailing_Lists/Searchbefore
>>  posting!
>> * Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists
>>
>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi Albert,

Apologies for hijacking your thread. Do you happen to have Fedora 17 as
well?

--
Szilárd


On Sun, Nov 4, 2012 at 10:55 AM, Albert  wrote:

> hello:
>
>  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x
> 1344 CUDA cores), and I got the following warnings:
>
> thank you very much.
>
> ---**messages--**-
>
> WARNING: On node 0: oversubscribing the available 0 logical CPU cores per
> node with 2 MPI processes.
>  This will cause considerable performance loss!
>
> 2 GPUs detected on host boreas:
>   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> compatible
>   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> compatible
>
> 2 GPUs auto-selected to be used for this run: #0, #1
>
> Using CUDA 8x8x8 non-bonded kernels
> Making 1D domain decomposition 1 x 2 x 1
>
> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> We have just committed the new CPU detection code in this branch,
> and will commit new SSE/AVX kernels in a few days. However, this
> means that currently only the NxN kernels are accelerated!
> In the mean time, you might want to avoid production runs in 4.6.
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi Thomas,

The output you get means that you don't have any of the macros we try to
use although your man pages seem to be referring to them. Hence, I'm really
clueless why is this happening. Could you please file a bug report on
redmine.gromacs.org and add both the initial output as well as my patch and
the resulting output. Don't forget to specify version of software you were
using.

Thanks,
--
Szilárd

On Thu, Nov 15, 2012 at 3:53 PM, Thomas Evangelidis wrote:

> Hi Szilárd,
>
> This is the warning message I get this time:
>
> WARNING: Oversubscribing the available -66 logical CPU cores with 1
> thread-MPI threads.
>
>  This will cause considerable performance loss!
>
> I have also attached the md.log file.
>
> thanks,
> Thomas
>
>
>
> On 14 November 2012 19:48, Szilárd Páll  wrote:
>
>> Hi Thomas,
>>
>> Could you please try applying the attached patch (git apply
>> hardware_detect.patch in the 4.6 source root) and let me know what the
>> output is?
>>
>> This should show which sysconf macro is used and what its return value is
>> as well as indicate if none of the macros are in fact defined by your
>> headers.
>>
>> Thanks,
>>
>> --
>> Szilárd
>>
>>
>>
>> On Sat, Nov 10, 2012 at 5:24 PM, Thomas Evangelidis wrote:
>>
>>>
>>>
>>> On 10 November 2012 03:21, Szilárd Páll  wrote:
>>>
 Hi,

 You must have an odd sysconf version! Could you please check what is
 the sysconf system variable's name in the sysconf man page (man sysconf)
 where it says something like:

 _SC_NPROCESSORS_ONLN
  The number of processors currently online.

 The first line should be one of the
 following: _SC_NPROCESSORS_ONLN, _SC_NPROC_ONLN,
 _SC_NPROCESSORS_CONF, _SC_NPROC_CONF, but I guess yours is something
 different.

>>>
>>> The following text is taken from man sysconf:
>>>
>>>These values also exist, but may not be standard.
>>>
>>> - _SC_PHYS_PAGES
>>>   The number of pages of physical memory.  Note that it is
>>> possible for the product of this value and the value of _SC_PAGE_SIZE to
>>> overflow.
>>>
>>> - _SC_AVPHYS_PAGES
>>>   The number of currently available pages of physical memory.
>>>
>>> - _SC_NPROCESSORS_CONF
>>>   The number of processors configured.
>>>
>>> - _SC_NPROCESSORS_ONLN
>>>   The number of processors currently online (available).
>>>
>>>
>>>
>>>
 Can you also check what your glibc version is?

>>>
>>> $ yum list installed | grep glibc
>>> glibc.i6862.15-57.fc17
>>> @updates
>>> glibc.x86_64  2.15-57.fc17
>>> @updates
>>> glibc-common.x86_64   2.15-57.fc17
>>> @updates
>>> glibc-devel.i686  2.15-57.fc17
>>> @updates
>>> glibc-devel.x86_642.15-57.fc17
>>> @updates
>>> glibc-headers.x86_64  2.15-57.fc17
>>> @updates
>>>
>>>
>>>


 On Fri, Nov 9, 2012 at 5:51 PM, Thomas Evangelidis 
 wrote:

>
>
>
> > I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
>> > benchmark with the following command line:
>> >
>> > mdrun_intel_cuda5 -v -s topol.tpr -testverlet
>> >
>> > "WARNING: Oversubscribing the available 0 logical CPU cores with 1
>> > thread-MPI threads."
>> >
>> > 0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core
>> i7-3610QM
>> >
>>
>> That is bizzarre. Could you run with "-debug 1" and have a look at the
>> mdrun.debug output which should contain a message like:
>> "Detected N processors, will use this as the number of supported
>> hardware
>> threads."
>>
>> I'm wondering, is N=0 in your case!?
>>
>> It says "Detected 0 processors, will use this as the number of
> supported hardware threads."
>
>
>>
>> > (2.3 GHz). Unlike Albert, I don't see any performance loss, I get
>> 13.4
>> > ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS
>> v4.5.5 on 4
>> > cores (8 threads) without the GPU. Yet, I don't see any performance
>> gain
>> > with more that 4 -nt threads.
>> >
>> > mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
>> > mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
>> > mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
>> > mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
>> > mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day
>> >
>>
>> I guess there is not much point in not using all cores, is it? Note
>> that
>> the performance drops after 4 threads because Hyper-Threading with
>> OpenMP
>> doesn't always help.
>>
>>
>> >
>> > I have also attached my log file (from "mdrun_intel_cuda5 -v -s
>> topol.tpr
>> > -testverlet") in case you find it helpf

Re: [gmx-users] GPU warnings

[Justin Lemkul]

On 11/15/12 9:53 AM, Thomas Evangelidis wrote:

Hi Szilárd,

This is the warning message I get this time:

WARNING: Oversubscribing the available -66 logical CPU cores with 1
thread-MPI threads.
  This will cause considerable performance loss!

I have also attached the md.log file.



Attachments are rejected by the mailing list.  They either have to be copied and 
pasted, linked, or sent to an individual specifically off-list.


-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi Thomas,

Could you please try applying the attached patch (git apply
hardware_detect.patch in the 4.6 source root) and let me know what the
output is?

This should show which sysconf macro is used and what its return value is
as well as indicate if none of the macros are in fact defined by your
headers.

Thanks,

--
Szilárd


On Sat, Nov 10, 2012 at 5:24 PM, Thomas Evangelidis wrote:

>
>
> On 10 November 2012 03:21, Szilárd Páll  wrote:
>
>> Hi,
>>
>> You must have an odd sysconf version! Could you please check what is the
>> sysconf system variable's name in the sysconf man page (man sysconf) where
>> it says something like:
>>
>> _SC_NPROCESSORS_ONLN
>>  The number of processors currently online.
>>
>> The first line should be one of the
>> following: _SC_NPROCESSORS_ONLN, _SC_NPROC_ONLN,
>> _SC_NPROCESSORS_CONF, _SC_NPROC_CONF, but I guess yours is something
>> different.
>>
>
> The following text is taken from man sysconf:
>
>These values also exist, but may not be standard.
>
> - _SC_PHYS_PAGES
>   The number of pages of physical memory.  Note that it is
> possible for the product of this value and the value of _SC_PAGE_SIZE to
> overflow.
>
> - _SC_AVPHYS_PAGES
>   The number of currently available pages of physical memory.
>
> - _SC_NPROCESSORS_CONF
>   The number of processors configured.
>
> - _SC_NPROCESSORS_ONLN
>   The number of processors currently online (available).
>
>
>
>
>> Can you also check what your glibc version is?
>>
>
> $ yum list installed | grep glibc
> glibc.i6862.15-57.fc17
> @updates
> glibc.x86_64  2.15-57.fc17
> @updates
> glibc-common.x86_64   2.15-57.fc17
> @updates
> glibc-devel.i686  2.15-57.fc17
> @updates
> glibc-devel.x86_642.15-57.fc17
> @updates
> glibc-headers.x86_64  2.15-57.fc17   @updates
>
>
>
>>
>>
>> On Fri, Nov 9, 2012 at 5:51 PM, Thomas Evangelidis wrote:
>>
>>>
>>>
>>>
>>> > I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
 > benchmark with the following command line:
 >
 > mdrun_intel_cuda5 -v -s topol.tpr -testverlet
 >
 > "WARNING: Oversubscribing the available 0 logical CPU cores with 1
 > thread-MPI threads."
 >
 > 0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core
 i7-3610QM
 >

 That is bizzarre. Could you run with "-debug 1" and have a look at the
 mdrun.debug output which should contain a message like:
 "Detected N processors, will use this as the number of supported
 hardware
 threads."

 I'm wondering, is N=0 in your case!?

 It says "Detected 0 processors, will use this as the number of
>>> supported hardware threads."
>>>
>>>

 > (2.3 GHz). Unlike Albert, I don't see any performance loss, I get 13.4
 > ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS
 v4.5.5 on 4
 > cores (8 threads) without the GPU. Yet, I don't see any performance
 gain
 > with more that 4 -nt threads.
 >
 > mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
 > mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
 > mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
 > mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
 > mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day
 >

 I guess there is not much point in not using all cores, is it? Note that
 the performance drops after 4 threads because Hyper-Threading with
 OpenMP
 doesn't always help.


 >
 > I have also attached my log file (from "mdrun_intel_cuda5 -v -s
 topol.tpr
 > -testverlet") in case you find it helpful.
 >

 I don't see it attached.



>>> I have attached both mdrun_intel_cuda5.debug and md.log files.  They
>>> will possibly be filtered by the mailing list but will be delivered to your
>>> email.
>>>
>>> thanksm
>>> Thomas
>>>
>>
>>
>
>
> --
>
> ==
>
> Thomas Evangelidis
>
> PhD student
> University of Athens
> Faculty of Pharmacy
> Department of Pharmaceutical Chemistry
> Panepistimioupoli-Zografou
> 157 71 Athens
> GREECE
>
> email: tev...@pharm.uoa.gr
>
>   teva...@gmail.com
>
>
> website: https://sites.google.com/site/thomasevangelidishomepage/
>
>
>
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

On 10 November 2012 03:21, Szilárd Páll  wrote:

> Hi,
>
> You must have an odd sysconf version! Could you please check what is the
> sysconf system variable's name in the sysconf man page (man sysconf) where
> it says something like:
>
> _SC_NPROCESSORS_ONLN
>  The number of processors currently online.
>
> The first line should be one of the
> following: _SC_NPROCESSORS_ONLN, _SC_NPROC_ONLN,
> _SC_NPROCESSORS_CONF, _SC_NPROC_CONF, but I guess yours is something
> different.
>

The following text is taken from man sysconf:

   These values also exist, but may not be standard.

- _SC_PHYS_PAGES
  The number of pages of physical memory.  Note that it is
possible for the product of this value and the value of _SC_PAGE_SIZE to
overflow.

- _SC_AVPHYS_PAGES
  The number of currently available pages of physical memory.

- _SC_NPROCESSORS_CONF
  The number of processors configured.

- _SC_NPROCESSORS_ONLN
  The number of processors currently online (available).




> Can you also check what your glibc version is?
>

$ yum list installed | grep glibc
glibc.i6862.15-57.fc17
@updates
glibc.x86_64  2.15-57.fc17
@updates
glibc-common.x86_64   2.15-57.fc17
@updates
glibc-devel.i686  2.15-57.fc17
@updates
glibc-devel.x86_642.15-57.fc17
@updates
glibc-headers.x86_64  2.15-57.fc17   @updates



>
>
> On Fri, Nov 9, 2012 at 5:51 PM, Thomas Evangelidis wrote:
>
>>
>>
>>
>> > I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
>>> > benchmark with the following command line:
>>> >
>>> > mdrun_intel_cuda5 -v -s topol.tpr -testverlet
>>> >
>>> > "WARNING: Oversubscribing the available 0 logical CPU cores with 1
>>> > thread-MPI threads."
>>> >
>>> > 0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core i7-3610QM
>>> >
>>>
>>> That is bizzarre. Could you run with "-debug 1" and have a look at the
>>> mdrun.debug output which should contain a message like:
>>> "Detected N processors, will use this as the number of supported hardware
>>> threads."
>>>
>>> I'm wondering, is N=0 in your case!?
>>>
>>> It says "Detected 0 processors, will use this as the number of supported
>> hardware threads."
>>
>>
>>>
>>> > (2.3 GHz). Unlike Albert, I don't see any performance loss, I get 13.4
>>> > ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS v4.5.5
>>> on 4
>>> > cores (8 threads) without the GPU. Yet, I don't see any performance
>>> gain
>>> > with more that 4 -nt threads.
>>> >
>>> > mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
>>> > mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
>>> > mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
>>> > mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
>>> > mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day
>>> >
>>>
>>> I guess there is not much point in not using all cores, is it? Note that
>>> the performance drops after 4 threads because Hyper-Threading with OpenMP
>>> doesn't always help.
>>>
>>>
>>> >
>>> > I have also attached my log file (from "mdrun_intel_cuda5 -v -s
>>> topol.tpr
>>> > -testverlet") in case you find it helpful.
>>> >
>>>
>>> I don't see it attached.
>>>
>>>
>>>
>> I have attached both mdrun_intel_cuda5.debug and md.log files.  They will
>> possibly be filtered by the mailing list but will be delivered to your
>> email.
>>
>> thanksm
>> Thomas
>>
>
>


-- 

==

Thomas Evangelidis

PhD student
University of Athens
Faculty of Pharmacy
Department of Pharmaceutical Chemistry
Panepistimioupoli-Zografou
157 71 Athens
GREECE

email: tev...@pharm.uoa.gr

  teva...@gmail.com


website: https://sites.google.com/site/thomasevangelidishomepage/
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi,

You must have an odd sysconf version! Could you please check what is the
sysconf system variable's name in the sysconf man page (man sysconf) where
it says something like:

_SC_NPROCESSORS_ONLN
 The number of processors currently online.

The first line should be one of the
following: _SC_NPROCESSORS_ONLN, _SC_NPROC_ONLN,
_SC_NPROCESSORS_CONF, _SC_NPROC_CONF, but I guess yours is something
different.

Can you also check what your glibc version is?

Thanks,

--
Szilárd


On Fri, Nov 9, 2012 at 5:51 PM, Thomas Evangelidis wrote:

>
>
>
> > I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
>> > benchmark with the following command line:
>> >
>> > mdrun_intel_cuda5 -v -s topol.tpr -testverlet
>> >
>> > "WARNING: Oversubscribing the available 0 logical CPU cores with 1
>> > thread-MPI threads."
>> >
>> > 0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core i7-3610QM
>> >
>>
>> That is bizzarre. Could you run with "-debug 1" and have a look at the
>> mdrun.debug output which should contain a message like:
>> "Detected N processors, will use this as the number of supported hardware
>> threads."
>>
>> I'm wondering, is N=0 in your case!?
>>
>> It says "Detected 0 processors, will use this as the number of supported
> hardware threads."
>
>
>>
>> > (2.3 GHz). Unlike Albert, I don't see any performance loss, I get 13.4
>> > ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS v4.5.5
>> on 4
>> > cores (8 threads) without the GPU. Yet, I don't see any performance gain
>> > with more that 4 -nt threads.
>> >
>> > mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
>> > mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
>> > mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
>> > mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
>> > mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day
>> >
>>
>> I guess there is not much point in not using all cores, is it? Note that
>> the performance drops after 4 threads because Hyper-Threading with OpenMP
>> doesn't always help.
>>
>>
>> >
>> > I have also attached my log file (from "mdrun_intel_cuda5 -v -s
>> topol.tpr
>> > -testverlet") in case you find it helpful.
>> >
>>
>> I don't see it attached.
>>
>>
>>
> I have attached both mdrun_intel_cuda5.debug and md.log files.  They will
> possibly be filtered by the mailing list but will be delivered to your
> email.
>
> thanksm
> Thomas
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

Hi,

On Tue, Nov 6, 2012 at 12:03 AM, Thomas Evangelidis wrote:

> Hi,
>
> I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
> benchmark with the following command line:
>
> mdrun_intel_cuda5 -v -s topol.tpr -testverlet
>
> "WARNING: Oversubscribing the available 0 logical CPU cores with 1
> thread-MPI threads."
>
> 0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core i7-3610QM
>

That is bizzarre. Could you run with "-debug 1" and have a look at the
mdrun.debug output which should contain a message like:
"Detected N processors, will use this as the number of supported hardware
threads."

I'm wondering, is N=0 in your case!?


> (2.3 GHz). Unlike Albert, I don't see any performance loss, I get 13.4
> ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS v4.5.5 on 4
> cores (8 threads) without the GPU. Yet, I don't see any performance gain
> with more that 4 -nt threads.
>
> mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
> mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
> mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
> mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
> mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day
>

I guess there is not much point in not using all cores, is it? Note that
the performance drops after 4 threads because Hyper-Threading with OpenMP
doesn't always help.


>
> I have also attached my log file (from "mdrun_intel_cuda5 -v -s topol.tpr
> -testverlet") in case you find it helpful.
>

I don't see it attached.

--
Szilárd


>
> Thanks,
> Thomas
>
>
>
> On 5 November 2012 18:54, Szilárd Páll  wrote:
>
> > The first warning indicates that you are starting more threads than the
> > hardware supports which would explain the poor performance.
> >
> > Could share a log file of the suspiciously slow run as well as the
> command
> > line you used to start mdrun?
> >
> > Cheers,
> >
> > --
> > Szilárd
> >
> >
> > On Sun, Nov 4, 2012 at 5:32 PM, Albert  wrote:
> >
> > > well, IC.
> > > the performance is rather poor than GTX590. 32ns/day vs 4 ns/day
> > > probably that's also something related to the warnings?
> > >
> > > THX
> > >
> > >
> > >
> > > On 11/04/2012 01:59 PM, Justin Lemkul wrote:
> > >
> > >>
> > >>
> > >> On 11/4/12 4:55 AM, Albert wrote:
> > >>
> > >>> hello:
> > >>>
> > >>>   I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti
> (2
> > x
> > >>> 1344
> > >>> CUDA cores), and I got the following warnings:
> > >>>
> > >>> thank you very much.
> > >>>
> > >>> ---**messages--**
> > >>> -
> > >>>
> > >>> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
> > >>> per node
> > >>> with 2 MPI processes.
> > >>>   This will cause considerable performance loss!
> > >>>
> > >>> 2 GPUs detected on host boreas:
> > >>>#0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> > >>> compatible
> > >>>#1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> > >>> compatible
> > >>>
> > >>> 2 GPUs auto-selected to be used for this run: #0, #1
> > >>>
> > >>> Using CUDA 8x8x8 non-bonded kernels
> > >>> Making 1D domain decomposition 1 x 2 x 1
> > >>>
> > >>> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> > >>> We have just committed the new CPU detection code in this branch,
> > >>> and will commit new SSE/AVX kernels in a few days. However, this
> > >>> means that currently only the NxN kernels are accelerated!
> > >>> In the mean time, you might want to avoid production runs in 4.6.
> > >>>
> > >>>
> > >> I can't address the first warning, but the second is fairly obvious.
> > >>  You're not using an official release, you're using the development
> > version
> > >> - let the user beware.  The code is not yet production-ready.
> > >>
> > >> -Justin
> > >>
> > >>
> > > --
> > > gmx-users mailing listgmx-users@gromacs.org
> > > http://lists.gromacs.org/**mailman/listinfo/gmx-users<
> > http://lists.gromacs.org/mailman/listinfo/gmx-users>
> > > * Please search the archive at http://www.gromacs.org/**
> > > Support/Mailing_Lists/Search<
> > http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
> > > * Please don't post (un)subscribe requests to the list. Use the www
> > > interface or send it to gmx-users-requ...@gromacs.org.
> > > * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
> > http://www.gromacs.org/Support/Mailing_Lists>
> > >
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
>
>
>
> --
>
> 

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

Hi,

I get these two warnings when I run the dhfr/GPU/dhfr-solv-PME.bench
benchmark with the following command line:

mdrun_intel_cuda5 -v -s topol.tpr -testverlet

"WARNING: Oversubscribing the available 0 logical CPU cores with 1
thread-MPI threads."

0 logical CPU cores? Isn't this bizarre? My CPU is Intel Core i7-3610QM
(2.3 GHz). Unlike Albert, I don't see any performance loss, I get 13.4
ns/day on a single core with 1 GPU and 13.2 ns/day with GROMACS v4.5.5 on 4
cores (8 threads) without the GPU. Yet, I don't see any performance gain
with more that 4 -nt threads.

mdrun_intel_cuda5 -v -nt 2 -s topol.tpr -testverlet : 15.4 ns/day
mdrun_intel_cuda5 -v -nt 3 -s topol.tpr -testverlet : 16.0 ns/day
mdrun_intel_cuda5 -v -nt 4 -s topol.tpr -testverlet : 16.3 ns/day
mdrun_intel_cuda5 -v -nt 6 -s topol.tpr -testverlet : 16.2 ns/day
mdrun_intel_cuda5 -v -nt 8 -s topol.tpr -testverlet : 15.4 ns/day

I have also attached my log file (from "mdrun_intel_cuda5 -v -s topol.tpr
-testverlet") in case you find it helpful.

Thanks,
Thomas



On 5 November 2012 18:54, Szilárd Páll  wrote:

> The first warning indicates that you are starting more threads than the
> hardware supports which would explain the poor performance.
>
> Could share a log file of the suspiciously slow run as well as the command
> line you used to start mdrun?
>
> Cheers,
>
> --
> Szilárd
>
>
> On Sun, Nov 4, 2012 at 5:32 PM, Albert  wrote:
>
> > well, IC.
> > the performance is rather poor than GTX590. 32ns/day vs 4 ns/day
> > probably that's also something related to the warnings?
> >
> > THX
> >
> >
> >
> > On 11/04/2012 01:59 PM, Justin Lemkul wrote:
> >
> >>
> >>
> >> On 11/4/12 4:55 AM, Albert wrote:
> >>
> >>> hello:
> >>>
> >>>   I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2
> x
> >>> 1344
> >>> CUDA cores), and I got the following warnings:
> >>>
> >>> thank you very much.
> >>>
> >>> ---**messages--**
> >>> -
> >>>
> >>> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
> >>> per node
> >>> with 2 MPI processes.
> >>>   This will cause considerable performance loss!
> >>>
> >>> 2 GPUs detected on host boreas:
> >>>#0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> >>> compatible
> >>>#1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
> >>> compatible
> >>>
> >>> 2 GPUs auto-selected to be used for this run: #0, #1
> >>>
> >>> Using CUDA 8x8x8 non-bonded kernels
> >>> Making 1D domain decomposition 1 x 2 x 1
> >>>
> >>> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
> >>> We have just committed the new CPU detection code in this branch,
> >>> and will commit new SSE/AVX kernels in a few days. However, this
> >>> means that currently only the NxN kernels are accelerated!
> >>> In the mean time, you might want to avoid production runs in 4.6.
> >>>
> >>>
> >> I can't address the first warning, but the second is fairly obvious.
> >>  You're not using an official release, you're using the development
> version
> >> - let the user beware.  The code is not yet production-ready.
> >>
> >> -Justin
> >>
> >>
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/**mailman/listinfo/gmx-users<
> http://lists.gromacs.org/mailman/listinfo/gmx-users>
> > * Please search the archive at http://www.gromacs.org/**
> > Support/Mailing_Lists/Search<
> http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
> > * Please don't post (un)subscribe requests to the list. Use the www
> > interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
> http://www.gromacs.org/Support/Mailing_Lists>
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>



-- 

==

Thomas Evangelidis

PhD student
University of Athens
Faculty of Pharmacy
Department of Pharmaceutical Chemistry
Panepistimioupoli-Zografou
157 71 Athens
GREECE

email: tev...@pharm.uoa.gr

  teva...@gmail.com


website: https://sites.google.com/site/thomasevangelidishomepage/
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Szilárd Páll]

The first warning indicates that you are starting more threads than the
hardware supports which would explain the poor performance.

Could share a log file of the suspiciously slow run as well as the command
line you used to start mdrun?

Cheers,

--
Szilárd


On Sun, Nov 4, 2012 at 5:32 PM, Albert  wrote:

> well, IC.
> the performance is rather poor than GTX590. 32ns/day vs 4 ns/day
> probably that's also something related to the warnings?
>
> THX
>
>
>
> On 11/04/2012 01:59 PM, Justin Lemkul wrote:
>
>>
>>
>> On 11/4/12 4:55 AM, Albert wrote:
>>
>>> hello:
>>>
>>>   I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x
>>> 1344
>>> CUDA cores), and I got the following warnings:
>>>
>>> thank you very much.
>>>
>>> ---**messages--**
>>> -
>>>
>>> WARNING: On node 0: oversubscribing the available 0 logical CPU cores
>>> per node
>>> with 2 MPI processes.
>>>   This will cause considerable performance loss!
>>>
>>> 2 GPUs detected on host boreas:
>>>#0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>>> compatible
>>>#1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>>> compatible
>>>
>>> 2 GPUs auto-selected to be used for this run: #0, #1
>>>
>>> Using CUDA 8x8x8 non-bonded kernels
>>> Making 1D domain decomposition 1 x 2 x 1
>>>
>>> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
>>> We have just committed the new CPU detection code in this branch,
>>> and will commit new SSE/AVX kernels in a few days. However, this
>>> means that currently only the NxN kernels are accelerated!
>>> In the mean time, you might want to avoid production runs in 4.6.
>>>
>>>
>> I can't address the first warning, but the second is fairly obvious.
>>  You're not using an official release, you're using the development version
>> - let the user beware.  The code is not yet production-ready.
>>
>> -Justin
>>
>>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Albert]

well, IC.
the performance is rather poor than GTX590. 32ns/day vs 4 ns/day
probably that's also something related to the warnings?

THX


On 11/04/2012 01:59 PM, Justin Lemkul wrote:



On 11/4/12 4:55 AM, Albert wrote:

hello:

  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti 
(2 x 1344

CUDA cores), and I got the following warnings:

thank you very much.

---messages---

WARNING: On node 0: oversubscribing the available 0 logical CPU cores 
per node

with 2 MPI processes.
  This will cause considerable performance loss!

2 GPUs detected on host boreas:
   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat: 
compatible
   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat: 
compatible


2 GPUs auto-selected to be used for this run: #0, #1

Using CUDA 8x8x8 non-bonded kernels
Making 1D domain decomposition 1 x 2 x 1

* WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
We have just committed the new CPU detection code in this branch,
and will commit new SSE/AVX kernels in a few days. However, this
means that currently only the NxN kernels are accelerated!
In the mean time, you might want to avoid production runs in 4.6.



I can't address the first warning, but the second is fairly obvious.  
You're not using an official release, you're using the development 
version - let the user beware.  The code is not yet production-ready.


-Justin



--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Thomas Evangelidis]

I 'm also get the first warning ("oversubscribing the available...") and
see no obvious performance gain. Do you know how to avoid that?

thanks,
Thomas



On 4 November 2012 14:59, Justin Lemkul  wrote:

>
>
> On 11/4/12 4:55 AM, Albert wrote:
>
>> hello:
>>
>>   I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x
>> 1344
>> CUDA cores), and I got the following warnings:
>>
>> thank you very much.
>>
>> ---**messages--**
>> -
>>
>> WARNING: On node 0: oversubscribing the available 0 logical CPU cores per
>> node
>> with 2 MPI processes.
>>   This will cause considerable performance loss!
>>
>> 2 GPUs detected on host boreas:
>>#0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> compatible
>>#1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat:
>> compatible
>>
>> 2 GPUs auto-selected to be used for this run: #0, #1
>>
>> Using CUDA 8x8x8 non-bonded kernels
>> Making 1D domain decomposition 1 x 2 x 1
>>
>> * WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
>> We have just committed the new CPU detection code in this branch,
>> and will commit new SSE/AVX kernels in a few days. However, this
>> means that currently only the NxN kernels are accelerated!
>> In the mean time, you might want to avoid production runs in 4.6.
>>
>>
> I can't address the first warning, but the second is fairly obvious.
>  You're not using an official release, you're using the development version
> - let the user beware.  The code is not yet production-ready.
>
> -Justin
>
> --
> ==**==
>
> Justin A. Lemkul, Ph.D.
> Research Scientist
> Department of Biochemistry
> Virginia Tech
> Blacksburg, VA
> jalemkul[at]vt.edu | (540) 231-9080
> http://www.bevanlab.biochem.**vt.edu/Pages/Personal/justin
>
> ==**==
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>



-- 

==

Thomas Evangelidis

PhD student
University of Athens
Faculty of Pharmacy
Department of Pharmaceutical Chemistry
Panepistimioupoli-Zografou
157 71 Athens
GREECE

email: tev...@pharm.uoa.gr

  teva...@gmail.com


website: https://sites.google.com/site/thomasevangelidishomepage/
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU warnings

[Justin Lemkul]

On 11/4/12 4:55 AM, Albert wrote:

hello:

  I am running Gromacs 4.6 GPU on a workstation with two GTX 660 Ti (2 x 1344
CUDA cores), and I got the following warnings:

thank you very much.

---messages---

WARNING: On node 0: oversubscribing the available 0 logical CPU cores per node
with 2 MPI processes.
  This will cause considerable performance loss!

2 GPUs detected on host boreas:
   #0: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat: compatible
   #1: NVIDIA GeForce GTX 660 Ti, compute cap.: 3.0, ECC:  no, stat: compatible

2 GPUs auto-selected to be used for this run: #0, #1

Using CUDA 8x8x8 non-bonded kernels
Making 1D domain decomposition 1 x 2 x 1

* WARNING * WARNING * WARNING * WARNING * WARNING * WARNING *
We have just committed the new CPU detection code in this branch,
and will commit new SSE/AVX kernels in a few days. However, this
means that currently only the NxN kernels are accelerated!
In the mean time, you might want to avoid production runs in 4.6.



I can't address the first warning, but the second is fairly obvious.  You're not 
using an official release, you're using the development version - let the user 
beware.  The code is not yet production-ready.


-Justin

--


Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin


--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists