I am not able to picture your network architecture. Would it possible for you to post a network diagram?
LACP is generally used between switches so that there is no looping issues when you have two switches cascaded using multiple cables. In such a case, only 1 cable is active and the second one takes over only when the first CABLE FAILS and not the switch it self. Hope you haven't connected *just these 2 switches* using LACP. If so, you will have to change it as it won't give you desired effect. LACP is used for UPLINK cables and not for peer-to-peer connectivity. With just 2 Switches, the right thing to do is 'stacking'. And one shouldn't use LACP on a Server-Switch connectivity. Normal non-LACP bonding is the correct way to do it. Will be to tell you more if you could post a diagram. Even a rough one would do. Indivar Nair On Thu, Jun 27, 2013 at 12:28 PM, Alfonso Pardo <alfonso.pa...@ciemat.es>wrote: > I would have high-availability if I have a bonding mode 0,1 or 2. But > I have LACP bonding attached to the same switch, if I loose a switch, my > OSS will be down. > > *From:* Indivar Nair <indivar.n...@techterra.in> > *Sent:* Thursday, June 27, 2013 8:30 AM > *To:* Alfonso Pardo <alfonso.pa...@ciemat.es> > *Cc:* Michael Shuey <sh...@purdue.edu> ; > lustre-discuss<lustre-discuss@lists.lustre.org> > *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces > > Then follow the instructions in my earlier mail. > No need to have bond0 and bond1. > You will achieve high-availability even with one bonded interface. > > Cheers, > > > Indivar Nair > > > On Thu, Jun 27, 2013 at 11:55 AM, Alfonso Pardo > <alfonso.pa...@ciemat.es>wrote: > >> Yes I have two swtiches, one to the bond0 interface and other switch >> to the second bond1 interface. >> >> >> >> *From:* Indivar Nair <indivar.n...@techterra.in> >> *Sent:* Wednesday, June 26, 2013 8:05 PM >> *To:* Alfonso Pardo <alfonso.pa...@ciemat.es> >> *Cc:* Michael Shuey <sh...@purdue.edu> ; >> WC-Discuss<wc-discuss.migrat...@intel.com>; >> lustre-discuss <lustre-discuss@lists.lustre.org> >> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces >> >> Hi Alfonso, >> >> I guess, you have two switches, with 2 interfaces (bond0) connected to >> one switch and the other 2 interfaces (bond1) to the second switch. >> >> --- >> >> What you need to do is merge the switches using a 'stacking' cable (if >> the switches are stackable) and create a single trunk using 2 ports from >> each switch. >> Then create a single bond on the Linux side using all the 4 Interfaces >> (and have just 1 IP). >> >> Use bonding mode balance-rr or 0 without LACP to get load balancing >> across all the 4 NICs. >> >> If the switches aren't stackable and a single trunk cannot be created on >> the switch side, then use bonding mode balance-alb or 6 on the Linux side. >> >> No changes need to be done to the cabling in either case. >> >> --- >> >> This way you get Load Balancing and H/A across NICs. >> >> >> >> Indivar Nair >> >> >> >> >> On Wed, Jun 26, 2013 at 5:47 PM, Michael Shuey <sh...@purdue.edu> wrote: >> >>> That will probably be slow - the machine you use to proxy the IPVS >>> address would be a bottleneck. Out of curiosity, what problem are you >>> trying to solve here? Do you anticipate whole-subnet outages to be an >>> issue (and if so, why)? >>> >>> -- >>> Mike Shuey >>> >>> >>> On Wed, Jun 26, 2013 at 4:53 AM, Alfonso Pardo >>> <alfonso.pa...@ciemat.es>wrote: >>> >>>> oooh! >>>> >>>> >>>> Thanks for you reply! May be another way is a floating IP between two >>>> interfaces with IPVS (corosync). >>>> >>>> -----Mensaje original----- From: Brian O'Connor >>>> Sent: Wednesday, June 26, 2013 10:15 AM >>>> To: Alfonso Pardo >>>> Cc: 'Michael Shuey' ; 'WC-Discuss' ; >>>> lustre-discuss@lists.lustre.**org<lustre-discuss@lists.lustre.org> >>>> Subject: Re: [Lustre-discuss] Lustre over two TCP interfaces >>>> >>>> >>>> >>>> >>>> >>>> On 06/26/2013 04:16 PM, Alfonso Pardo wrote: >>>> >>>>> But.... if I configure the OST assigning to the first interface of the >>>>> OSS (bond0) and as failover OSS the second inteface of the OSS. If the >>>>> bond0 network down, the client will try to connect to the failover, >>>>> that >>>>> is the second interface of the OSS. >>>>> is it possible? >>>>> >>>> >>>> >>>> I stand to be corrected, but no, I don't think so. As I understand it >>>> the failover code looks for a different server instance, rather than a >>>> different nid. >>>> >>>> See >>>> >>>> http://lists.opensfs.org/**pipermail/lustre-devel-** >>>> opensfs.org/2012-August/**000028.html<http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html> >>>> >>>> >>>> *From:* Brian O'Connor <mailto:bri...@sgi.com> >>>>> *Sent:* Wednesday, June 26, 2013 1:09 AM >>>>> *To:* 'Alfonso Pardo' >>>>> <mailto:alfonso.pardo@ciemat.**es<alfonso.pa...@ciemat.es>> >>>>> ; 'Michael Shuey' >>>>> <mailto:sh...@purdue.edu> >>>>> *Cc:* 'WC-Discuss' >>>>> <mailto:WC-Discuss.Migration@**intel.com<wc-discuss.migrat...@intel.com>> >>>>> ; >>>>> mailto:lustre-discuss@lists.**lustre.org<lustre-discuss@lists.lustre.org> >>>>> *Subject:* RE: [Lustre-discuss] Lustre over two TCP interfaces >>>>> Unless something has changed in the new versions of lustre, I don't >>>>> think lustre can do failover between nids on the same machine. >>>>> >>>>> It can choose the available nid at mount time, but if an active nid >>>>> goes >>>>> away after you are mounted then the client chooses the failover nid, >>>>> and >>>>> this must be on a different server. >>>>> >>>>> Check the archives for more discussion in this topic :) >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> *From: *Alfonso Pardo [alfonso.pa...@ciemat.es >>>>> <mailto:alfonso.pardo@ciemat.**es <alfonso.pa...@ciemat.es>>] >>>>> *Sent: *Tuesday, June 25, 2013 07:23 AM Central Standard Time >>>>> *To: *Michael Shuey >>>>> *Cc: *WC-Discuss; >>>>> lustre-discuss@lists.lustre.**org<lustre-discuss@lists.lustre.org> >>>>> *Subject: *Re: [Lustre-discuss] Lustre over two TCP interfaces >>>>> >>>>> thank Michael, >>>>> This is my second step, I will change the lnet with “options lnet >>>>> networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have a >>>>> bond0 and bond1 with LACP. I need to comunicate the clients with two >>>>> network for HA network. >>>>> If the bond0 network is down, the clients can reach the OSS by the >>>>> second network bond1. >>>>> If I change the modprobe with “options lnet >>>>> networks=tcp0(bond0),tcp1(**bond1)”, how the clients mount the >>>>> filesystem >>>>> to reach the OSS by two network? >>>>> *From:* Michael Shuey <mailto:sh...@purdue.edu> >>>>> *Sent:* Tuesday, June 25, 2013 2:14 PM >>>>> *To:* Alfonso Pardo >>>>> <mailto:alfonso.pardo@ciemat.**es<alfonso.pa...@ciemat.es> >>>>> > >>>>> *Cc:* lustre-discuss@lists.lustre.**org<lustre-discuss@lists.lustre.org> >>>>> <mailto:lustre-discuss@lists.**lustre.org<lustre-discuss@lists.lustre.org>> >>>>> ; WC-Discuss >>>>> <mailto:WC-Discuss.Migration@**intel.com<wc-discuss.migrat...@intel.com> >>>>> > >>>>> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces >>>>> Different interfaces need to be declared with different LNET networks - >>>>> something like "networks=tcp0(eth0),tcp1(**eth1)". Of course, that >>>>> assumes your clients are configured to use a mix of tcp0 and tcp1 for >>>>> connections (with each client only using one of the two). This is >>>>> really only useful in corner cases, when you're doing something >>>>> strange; >>>>> if eth0 and eth1 are in the same subnet (as in your example), this is >>>>> almost certainly not productive. >>>>> A better bet might be to use a single LNET, and bond the two interfaces >>>>> together - either as an active/passive pair, or active/active (e.g., >>>>> LACP). Then you'd declare networks=tcp0(bond0), give the bond a single >>>>> IP address, and client traffic would be split across the two members in >>>>> the bond more like you probably expect (given the limits of the bond >>>>> protocol you're using). >>>>> -- >>>>> Mike Shuey >>>>> >>>>> >>>>> On Tue, Jun 25, 2013 at 8:06 AM, Alfonso Pardo < >>>>> alfonso.pa...@ciemat.es >>>>> <mailto:alfonso.pardo@ciemat.**es <alfonso.pa...@ciemat.es>>> wrote: >>>>> >>>>> hello friends, >>>>> I need to comunicate my OSS by two ethernet TCP interfaces: eth0 >>>>> and >>>>> eth1. >>>>> I have configured this feature in my modprobe.d with: >>>>> “options lnet networks=tcp0(eth0,eth1)” >>>>> And I can see two interfaces with: >>>>> lctl --net tcp interface_list >>>>> sa-d4-01.ceta-ciemat.es >>>>> <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es> >>>>> >: >>>>> (192.168.11.15/255.255.255.0 >>>>> <http://192.168.11.15/255.255.**255.0<http://192.168.11.15/255.255.255.0> >>>>> >) >>>>> npeer 0 nroute 2 >>>>> sa-d4-01.ceta-ciemat.es >>>>> <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es> >>>>> >: >>>>> (192.168.11.35/255.255.255.0 >>>>> <http://192.168.11.35/255.255.**255.0<http://192.168.11.35/255.255.255.0> >>>>> >) >>>>> npeer 0 nroute 0 >>>>> But, the clients only can communicate with the first interface: >>>>> lctl ping 192.168.11.15 >>>>> 12345-0@lo >>>>> 12345-192.168.11.15@tcp >>>>> lctl ping 192.168.11.35 >>>>> failed to ping 192.168.11.35@tcp: Input/output error >>>>> Any suggestions how to “enable” the second interface? >>>>> thank in advance >>>>> >>>>> /Alfonso Pardo Diaz/ >>>>> /*System Administrator / Researcher*/ >>>>> /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/ >>>>> /Tel: +34 927 65 93 17 <tel:%2B34%20927%2065%2093%**2017> Fax: >>>>> +34 927 >>>>> 32 32 37/ >>>>> >>>>> CETA-Ciemat logo <http://www.ceta-ciemat.es/> >>>>> >>>>> ---------------------------- Confidencialidad: Este mensaje y sus >>>>> ficheros adjuntos se dirige exclusivamente a su destinatario y >>>>> puede >>>>> contener información privilegiada o confidencial. Si no es vd. el >>>>> destinatario indicado, queda notificado de que la utilización, >>>>> divulgación y/o copia sin autorización está prohibida en virtud de >>>>> la legislación vigente. Si ha recibido este mensaje por error, le >>>>> rogamos que nos lo comunique inmediatamente respondiendo al mensaje >>>>> y proceda a su destrucción. Disclaimer: This message and its >>>>> attached files is intended exclusively for its recipients and may >>>>> contain confidential information. If you received this e-mail in >>>>> error you are hereby notified that any dissemination, copy or >>>>> disclosure of this communication is strictly prohibited and may be >>>>> unlawful. In this case, please notify us by a reply and delete this >>>>> email and its contents immediately. ---------------------------- >>>>> >>>>> ______________________________**_________________ >>>>> Lustre-discuss mailing list >>>>> >>>>> Lustre-discuss@lists.lustre.**org<Lustre-discuss@lists.lustre.org><mailto: >>>>> Lustre-discuss@lists.**lustre.org <Lustre-discuss@lists.lustre.org>> >>>>> >>>>> http://lists.lustre.org/**mailman/listinfo/lustre-**discuss<http://lists.lustre.org/mailman/listinfo/lustre-discuss> >>>>> >>>>> >>>> >>>> -- >>>> Brian O'Connor >>>> ------------------------------**------------------------------**- >>>> SGI Consulting >>>> Email: bri...@sgi.com, Mobile +61 417 746 452 <%2B61%20417%20746%20452> >>>> Phone: +61 3 9963 1900, Fax: +61 3 9963 1902 <%2B61%203%209963%201902> >>>> 691 Burke Road, Camberwell, Victoria, 3124 >>>> AUSTRALIA >>>> http://www.sgi.com/support/**services<http://www.sgi.com/support/services> >>>> ------------------------------**------------------------------**- >>>> >>> >>> >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss@lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>> >>> >> > >
_______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
