The way I linked the two machines was to have them on a different subnet. For example, their regular ethernet on the LAN might be 192.168.0.100 and 192.168.0.101. Then for the IPoIB configuration, the machines would be setup as 192.168.10.100 and 192.168.10.101. Totally isolated from the LAN, and we would set up the software we were clustering with to find its neighbor node using that latter IP setup. It worked most of the time, especially after a fresh reboot. Even with the IP adding a little extra overhead (I didnt know any other way to set the software up) those cards are extremely fast. Phillip Partipilo Parametric Solutions Inc. Jupiter, Florida (561) 747-6107
_____ From: Adam Meixler [mailto:ad...@interlink1.com] Sent: Tuesday, September 29, 2009 3:43 PM To: NT System Admin Issues Subject: RE: infiniband Thanks! It is a little scary to hear a core piece like openSM was flaky but we'll keep an eye out for that now. The whole concept of subnet management does elude me at the moment though. From: Phillip Partipilo [mailto:p...@psnet.com] Sent: Tuesday, September 29, 2009 1:29 PM To: NT System Admin Issues Subject: RE: infiniband I put together a two-system cluster with a couple 4x Mellanox cards several years ago, kind of tinkering around to connect two dualcore/dualsocket opteron boxes with IPoIB. Systems ran Fedora Core 5, 2.6.20-1.2320.fc5. Used some something called OpenIB software if I recall correctly. It was flaky, this "opensm" process would sometimes shoot to eating up 100% CPU for no good reason, when there wasnt even any processes actively using the IB link. I may have done something wrong . Anyway nobody really used it, and now we have a machine faster then those two nodes together so the concept has been scrapped. I bet with a fresher version of Linux it would have been more successful. Phillip Partipilo Parametric Solutions Inc. Jupiter, Florida (561) 747-6107 _____ From: Adam Meixler [mailto:ad...@interlink1.com] Sent: Tuesday, September 29, 2009 1:06 PM To: NT System Admin Issues Subject: infiniband hi, I was just curious if anyone has spent any time with InfiniBand, more specifically IPoIB for use with iSCSI. We have our eye on the Mellanox hardware to cluster build with but are interested in any stories anyone has on deploying infiniband. thanks! _____ If this email is spam, report it here: <http://www.onlymyemail.com/view/?action=reportSpam&Id=ODEzNjQ6OTcxNzIwMzk0O nBqcEBwc25ldC5jb20%3D> http://www.OnlyMyEmail.com/ReportSpam THIS ELECTRONIC MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL AND PROPRIETARY PROPERTY OF THE SENDER. THE INFORMATION IS INTENDED FOR USE BY THE ADDRESSEE ONLY. ANY OTHER INTERCEPTION, COPYING, ACCESSING, OR DISCLOSURE OF THIS MESSAGE IS PROHIBITED. IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR, PLEASE IMMEDIATELY NOTIFY THE SENDER AND DELETE THIS MAIL AND ALL ATTACHMENTS. DO NOT FORWARD THIS MESSAGE WITHOUT PERMISSION OF THE SENDER. THIS ELECTRONIC MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL AND PROPRIETARY PROPERTY OF THE SENDER. THE INFORMATION IS INTENDED FOR USE BY THE ADDRESSEE ONLY. ANY OTHER INTERCEPTION, COPYING, ACCESSING, OR DISCLOSURE OF THIS MESSAGE IS PROHIBITED. IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR, PLEASE IMMEDIATELY NOTIFY THE SENDER AND DELETE THIS MAIL AND ALL ATTACHMENTS. DO NOT FORWARD THIS MESSAGE WITHOUT PERMISSION OF THE SENDER. _____ If this email is spam, report it here: http://www.OnlyMyEmail.com/ReportSpam <http://www.onlymyemail.com/view/?action=reportSpam&Id=ODEzNjQ6OTcxODIxMzI3O nBqcEBwc25ldC5jb20%3D> THIS ELECTRONIC MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL AND PROPRIETARY PROPERTY OF THE SENDER. THE INFORMATION IS INTENDED FOR USE BY THE ADDRESSEE ONLY. ANY OTHER INTERCEPTION, COPYING, ACCESSING, OR DISCLOSURE OF THIS MESSAGE IS PROHIBITED. IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR, PLEASE IMMEDIATELY NOTIFY THE SENDER AND DELETE THIS MAIL AND ALL ATTACHMENTS. DO NOT FORWARD THIS MESSAGE WITHOUT PERMISSION OF THE SENDER. THIS ELECTRONIC MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL AND PROPRIETARY PROPERTY OF THE SENDER. THE INFORMATION IS INTENDED FOR USE BY THE ADDRESSEE ONLY. ANY OTHER INTERCEPTION, COPYING, ACCESSING, OR DISCLOSURE OF THIS MESSAGE IS PROHIBITED. IF YOU HAVE RECEIVED THIS MESSAGE IN ERROR, PLEASE IMMEDIATELY NOTIFY THE SENDER AND DELETE THIS MAIL AND ALL ATTACHMENTS. DO NOT FORWARD THIS MESSAGE WITHOUT PERMISSION OF THE SENDER. ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~
