Aw: Re: Re: [gmx-users] GPU-based workstation
Dear Dr. Pali, Thank you, Stephan Watkins Gesendet: Dienstag, 28. Mai 2013 um 19:50 Uhr Von: "Szilárd Páll" An: "Discussion list for GROMACS users" Betreff: Re: Re: [gmx-users] GPU-based workstation Dear all, As far as I understand, the OP is interested in hardware for *running* GROMACS 4.6 rather than developing code. or running LINPACK. To get best performance it is important to use a machine with hardware balanced for GROMACS' workloads. Too little GPU resources will result in CPU idling; too much GPU resources will lead to the runs being CPU or multi-GPU scaling bound and above a certain level GROMACS won't be able to make use of additional GPUs. Of course, the balance will depend both on hardware and simulation settings (mostly the LJ cut-off used). An additional factor to consider is typical system size. To reach near peak pair-force throughput on GPUs you typically need >20k-40k particles/GPU (depends on the architecture) and throughput drops below these values. Hence, in most cases it is preferred to use fewer and faster GPUs rather than more. Without knowing the budgdet and indented use of the machine it is hard to make suggestions, but I would say for a budget desktop box a quad-core Intel Ivy Bridge or the top-end AMD Piledriver CPU with a fast Kepler GTX card (e.g. GTX 680 or GTX 770/780) should work well. If you're considering dual-socket workstations, I suggest you go with the higher core-count and higher frequency Intel CPUs (6+ cores >2.2 GHz), otherwise you may not see as much benefit as you would expect based on the insane price tag (especially if you compare to an i7 3939K or its IVB successor). Cheers, -- Szilárd On Sat, May 25, 2013 at 1:02 PM, lloyd riggs wrote: > More RAM the better, and the best I have seen is 4 GPU work station. I can > use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is > really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems > though with GPU's, as on a board there packed, so extra cooling things may > help not blow a GPU, but I would look for good ones (ask around), as its a > video game market and they go for looks even though its in casing? The > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont > know performance wise, the onboard GPU, the more RAM the marrier...so yes, > normal work stations you can get 4 GPU's for a 300 US$ board, but then the > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad is > also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests > software, not Gromacs, so would be nice to see performance...for a small 100 > atom molecule and 500 solvent, using just the CPU I get it to run 5-10 > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000 solvent > eq (NVT or NPT) runs and they clock at around 1 hour real for say 50 ps > eq's > > Stephan > > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr > Von: "James Starlight" > An: "Discussion list for GROMACS users" > Betreff: Re: [gmx-users] GPU-based workstation > Dear Dr. Watkins! > > Thank you for the suggestions! > > In the local shops I've found only Core i7 with 6 cores (like Core > i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores > than with 4 cores in case of i7 cpu (assuming that I run simulation in > cpu+gpu mode )? > > Also you've mentioned about 4 PCeI MD. Does it means that modern > work-station could have 4 GPU's in one home-like desktop ? According to my > current task I suppose that 2 GPU's would be suitable for my simulations > (assuming that I use typical ASUS MB and 650 Watt power unit). Have > someone tried to use several GPU's on one workstation ? What attributes of > MB should be taken into account for best performance on such multi-gpu > station ? > > James > > 2013/5/25 lloyd riggs > >> There's also these, but 1 chip runs 6K US, they can get performance up to >> 2.3 teraflops per chip though double percission...but have no clue about >> integration with GPU's...Intell also sells their chips on PCIe cards...but >> get only about 350 Gflops, and run 1K US$. >> >> http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor >> http://www.xilinx.com/ >> >> They can design them though to fit a PCIe slot and run about the same, but >> still need the board, ram etc... >> >> Mostly just to dream about, they say you can order them with radiation >> shielding as well...so... >> >> Stephan Watkins >> >> *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr >> *Von:* "James Starlight" >> *An:* "Discussion list for GROMACS users" >> *Betreff:* [gmx-users] GPU-based workstation >> Dear Gromacs Users! >> >> >> I'd like to build new workstation for performing simulation on GPU with >> Gromacs 4.6 native cuda support. >> Recently I've used such setup with Core i5 cpu and nvidia 670 GTX video >> and obtain good performance ( ~ 20 ns\day for typic
Aw: Re: Re: [gmx-users] GPU-based workstation
You can also look at profilling on varied web sites, the high end Nvidia run only slightly better than the 2 year old ones, from an individual point not worth the money yet, but if you have the money? as I've been browsing. Also, the sim I did on the cluster was 180-190,000 atoms so the exact same performance the other person had. Stephan Gesendet: Samstag, 25. Mai 2013 um 15:19 Uhr Von: "James Starlight" An: "Discussion list for GROMACS users" Betreff: Re: Aw: Re: [gmx-users] GPU-based workstation Richard, thanks for suggestion! Assuming that I'm using 2 high end GeForce's what performance be better 1) in case of one i7 (4 or 6 nodes ) ? 2) in case of 8 core Xeon like CPU Intel Xeon E5-2650 2.0 GHz / 8core What properties of MB should take into account primarily for such Xenon-based system. Does such MBs support multi-GPU ( I noticed that many such MBs lack for PCI)? James 2013/5/25 Broadbent, Richard > I've been running on my Universities GPU nodes these are one E5-xeon > (6-cores 12 threads) and have 4 Nvidia 690gtx's. My system is 93 000 atoms > of DMF under NVE. The performance has been a little disappointing > ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I get > 5.4ns/day for the same system. On our HPC system using 32 nodes each with 2 > quad-core xeon processors I get 30-40ns/day. > > I think that to achieve reasonable performance the system has to be > balanced between CPU's and GPU's probably getting 2 high end GPU's and a > top end xeon E5 or core i7 would be a good choice. > > > Richard > > From: lloyd riggs > > Reply-To: Discussion users > gmx-users@gromacs.org>> > Date: Saturday, 25 May 2013 12:02 > To: Discussion users > > Subject: Aw: Re: [gmx-users] GPU-based workstation > > More RAM the better, and the best I have seen is 4 GPU work station. I > can use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is > really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems > though with GPU's, as on a board there packed, so extra cooling things may > help not blow a GPU, but I would look for good ones (ask around), as its a > video game market and they go for looks even though its in casing? The > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont > know performance wise, the onboard GPU, the more RAM the marrier...so yes, > normal work stations you can get 4 GPU's for a 300 US$ board, but then the > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad > is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests > software, not Gromacs, so would be nice to see performance...for a small > 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10 > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000 > solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say > 50 ps eq's > > Stephan > > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr > Von: "James Starlight" > jmsstarli...@gmail.com>> > An: "Discussion list for GROMACS users" > gmx-users@gromacs.org>> > Betreff: Re: [gmx-users] GPU-based workstation > Dear Dr. Watkins! > > Thank you for the suggestions! > > In the local shops I've found only Core i7 with 6 cores (like Core > i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores > than with 4 cores in case of i7 cpu (assuming that I run simulation in > cpu+gpu mode )? > > Also you've mentioned about 4 PCeI MD. Does it means that modern > work-station could have 4 GPU's in one home-like desktop ? According to my > current task I suppose that 2 GPU's would be suitable for my simulations > (assuming that I use typical ASUS MB and 650 Watt power unit). Have > someone tried to use several GPU's on one workstation ? What attributes of > MB should be taken into account for best performance on such multi-gpu > station ? > > James > > 2013/5/25 lloyd riggs > > > > There's also these, but 1 chip runs 6K US, they can get performance up to > > 2.3 teraflops per chip though double percission...but have no clue about > > integration with GPU's...Intell also sells their chips on PCIe > cards...but > > get only about 350 Gflops, and run 1K US$. > > > > http://en.wikipedia.org/wiki/Field-programmable_gate_array and vendor > > http://www.xilinx.com/ > > > > They can design them though to fit a PCIe slot and run about the same, > but > > still need the board, ram etc... > > > > Mostly just to dream about, they say you can order them with radiation > > shielding as well...so... > > > > Stephan Watkins > > > > *Gesendet:* Freitag, 24. Mai 2013 um 13:17 Uhr > > *Von:* "James Starlight" > jmsstarli...@gmail.com>> > > *An:* "Discussion list for GROMACS users" > gmx-users@gromacs.org>> > > *Betreff:* [gmx-users] GPU-based workstation > > Dear Gromacs Users! > > > > > > I'd like to build new workstation for performing simulation on GPU with
Aw: Re: Re: [gmx-users] GPU-based workstation
Id go for the i7 6 core, To the other message, funny. I bought ATI's as they clock faster and cost 1/3 the price of Nvidia's but then the software all went to Nvidia. The new ATI with twice the shaders runs at the same speed (around 1-1.3 terflops ) due to the same problems the Nvidias ran into with IO (or maybe onboard RAM does solve the problem if they went up to 16 or 32 MB) Gromacs, etc...doesn't run on ATI's, and I've been hoping they, AMD, catch up, but all I ever see is the constant in 6 months then nothing. I ran around 40 4 ns simulations on University blades with 8 AMD quad cores, using 3 blades I only was able to get 1 ns/day, but never pressed it as far as why so slow, as I needed to finish. With the Nvidia at even 5 ns/day I or alot of people could do some really nice work as far as publishing, with raw data in 2 weeks time, so now I feel a bit saddened... I also just found openCL profilling with CUDA 5 that will take any C or C++ software, and mark all sections you need to convert to openCL, but the trial software is 30 day, then 250 US$... Stephan Gesendet: Samstag, 25. Mai 2013 um 15:19 Uhr Von: "James Starlight" An: "Discussion list for GROMACS users" Betreff: Re: Aw: Re: [gmx-users] GPU-based workstation Richard, thanks for suggestion! Assuming that I'm using 2 high end GeForce's what performance be better 1) in case of one i7 (4 or 6 nodes ) ? 2) in case of 8 core Xeon like CPU Intel Xeon E5-2650 2.0 GHz / 8core What properties of MB should take into account primarily for such Xenon-based system. Does such MBs support multi-GPU ( I noticed that many such MBs lack for PCI)? James 2013/5/25 Broadbent, Richard > I've been running on my Universities GPU nodes these are one E5-xeon > (6-cores 12 threads) and have 4 Nvidia 690gtx's. My system is 93 000 atoms > of DMF under NVE. The performance has been a little disappointing > ~10ns/day. On my home system using a core i5-2500 and a nvidia 560ti I get > 5.4ns/day for the same system. On our HPC system using 32 nodes each with 2 > quad-core xeon processors I get 30-40ns/day. > > I think that to achieve reasonable performance the system has to be > balanced between CPU's and GPU's probably getting 2 high end GPU's and a > top end xeon E5 or core i7 would be a good choice. > > > Richard > > From: lloyd riggs > > Reply-To: Discussion users > gmx-users@gromacs.org>> > Date: Saturday, 25 May 2013 12:02 > To: Discussion users > > Subject: Aw: Re: [gmx-users] GPU-based workstation > > More RAM the better, and the best I have seen is 4 GPU work station. I > can use/have used 4. The GPU takes 2 slots though, so a 7-8 PCIe board is > really 3-4 GPU, except the tyan mentioned (there designed as blades so an 8 > or 10 slot board really holds 8 or 10 GPU's). There's cooling problems > though with GPU's, as on a board there packed, so extra cooling things may > help not blow a GPU, but I would look for good ones (ask around), as its a > video game market and they go for looks even though its in casing? The > external RAM (not onboard GPU RAM) helps if you do a larger sim, but I dont > know performance wise, the onboard GPU, the more RAM the marrier...so yes, > normal work stations you can get 4 GPU's for a 300 US$ board, but then the > price goes way up (3-4000 US$ for an 8-10 gpu board). RAM ordered abroad > is also cheep, 8 or 16 MB Vs. Shop...I have used 4 GPU's but only on tests > software, not Gromacs, so would be nice to see performance...for a small > 100 atom molecule and 500 solvent, using just the CPU I get it to run 5-10 > minutes real for 1 ns sim, but tried simple large 800 amino, 25,000 > solvent eq (NVT or NPT) runs and they clock at around 1 hour real for say > 50 ps eq's > > Stephan > > Gesendet: Samstag, 25. Mai 2013 um 07:54 Uhr > Von: "James Starlight" > jmsstarli...@gmail.com>> > An: "Discussion list for GROMACS users" > gmx-users@gromacs.org>> > Betreff: Re: [gmx-users] GPU-based workstation > Dear Dr. Watkins! > > Thank you for the suggestions! > > In the local shops I've found only Core i7 with 6 cores (like Core > i7-39xx) and 4 cores. Should I obtain much better performance with 6 cores > than with 4 cores in case of i7 cpu (assuming that I run simulation in > cpu+gpu mode )? > > Also you've mentioned about 4 PCeI MD. Does it means that modern > work-station could have 4 GPU's in one home-like desktop ? According to my > current task I suppose that 2 GPU's would be suitable for my simulations > (assuming that I use typical ASUS MB and 650 Watt power unit). Have > someone tried to use several GPU's on one workstation ? What attributes of > MB should be taken into account for best performance on such multi-gpu > station ? > > James > > 2013/5/25 lloyd riggs > > > > There's also these, but 1 chip runs 6K US, they can get performance up to > > 2.3 teraflops per chip though double percission...but have no clue about > > integration with GPU's...Intell also sells their chips on PCIe > cards...b
