Re: [Gluster-users] working inifiniband HW ??
On 08/07/2013, at 6:02 PM, Harry Mangalam wrote: > I concur - I used the same cards on our test cluster - just make sure you > upgrade the firmware to the latest revision: > http://www.mellanox.com/content/pages.php?pg=custom_firmware_table > > The fw upgrade is not trivial - well, it IS trivial, but bringing all the > info together was not - if you want some rough notes, let me know. Nowdays there should be copious info on the new Mellanox Community website (recently launched): http://community.mellanox.com Saying that because I've written out the step-by-step instructions for people on the forums there a bunch of times over the last month or two. :) Searching for "flint" or "mstflint" should pull up appropriate stuff for anyone else who needs it. Hopefully useful. :) Regards and best wishes, Justin Clift > the other harry -- Open Source and Standards @ Red Hat twitter.com/realjustinclift ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] working inifiniband HW ??
I concur - I used the same cards on our test cluster - just make sure you upgrade the firmware to the latest revision: http://www.mellanox.com/content/pages.php?pg=custom_firmware_table The fw upgrade is not trivial - well, it IS trivial, but bringing all the info together was not - if you want some rough notes, let me know. the other harry On Mon, Jul 8, 2013 at 5:23 AM, Justin Clift wrote: > On 08/07/2013, at 10:20 AM, HL wrote: > > I am currently testing glusterfs on a small scale non-production env > with ordinary nics. > > > > I would like to purchase a couple of inifiniband nics in order to > connect 3 servers in point to point mode > > that is with no switches in between. > > > > Since I've noticed that some of you have this kind of H/W > > Any info on brands/models and > > a "good known to work setup" > > will be highly appreciated. > > Depends on the kind of performance you're after. :) > > If you're just wanting something better than 10GbE at minimal cost, > these work fine under Linux: > > http://www.ebay.co.uk/itm/360657396651 > ~US$40 each, not counting postage > > They're rebadged Mellanox MHGH28-XTC cards. Completely ok to be > flashed with standard Mellanox firmware. > > For cables, I use these: > > http://www.ebay.co.uk/itm/251200441924 > US$14 each, not counting postage > > Note, I'm comfortable getting stuff off eBay where the seller > seems ok. So far, so good. ;) > > If you do decide to use the above stuff, and later on want a switch, > try and find a Voltaire ISR 9024D-M. They're DDR infiniband (20Gb/s > per port), and can run fanless (completely silent). > > http://www.ebay.co.uk/itm/350827346490 > > They can be had pretty cheaply if you're willing to wait for good > pricing. I got mine for ~US$270. > > Hope that helps. :) > > Regards and best wishes, > > Justin Clift > > > > Regards, > > Harry > > ___ > > Gluster-users mailing list > > Gluster-users@gluster.org > > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > -- > Open Source and Standards @ Red Hat > > twitter.com/realjustinclift > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users > -- Data backup by the NSA - your tax dollars at work. -- Harry Mangalam - Research Computing, OIT, Rm 225 MSTB, UC Irvine [m/c 2225] / 92697 Google Voice Multiplexer: (949) 478-4487 415 South Circle View Dr, Irvine, CA, 92697 [shipping] MSTB Lat/Long: (33.642025,-117.844414) (paste into Google Maps) ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] working inifiniband HW ??
On 08/07/2013, at 10:20 AM, HL wrote: > I am currently testing glusterfs on a small scale non-production env with > ordinary nics. > > I would like to purchase a couple of inifiniband nics in order to connect 3 > servers in point to point mode > that is with no switches in between. > > Since I've noticed that some of you have this kind of H/W > Any info on brands/models and > a "good known to work setup" > will be highly appreciated. Depends on the kind of performance you're after. :) If you're just wanting something better than 10GbE at minimal cost, these work fine under Linux: http://www.ebay.co.uk/itm/360657396651 ~US$40 each, not counting postage They're rebadged Mellanox MHGH28-XTC cards. Completely ok to be flashed with standard Mellanox firmware. For cables, I use these: http://www.ebay.co.uk/itm/251200441924 US$14 each, not counting postage Note, I'm comfortable getting stuff off eBay where the seller seems ok. So far, so good. ;) If you do decide to use the above stuff, and later on want a switch, try and find a Voltaire ISR 9024D-M. They're DDR infiniband (20Gb/s per port), and can run fanless (completely silent). http://www.ebay.co.uk/itm/350827346490 They can be had pretty cheaply if you're willing to wait for good pricing. I got mine for ~US$270. Hope that helps. :) Regards and best wishes, Justin Clift > Regards, > Harry > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users -- Open Source and Standards @ Red Hat twitter.com/realjustinclift ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
