> > For two links it should be quite good - it seemed to balance across > > that quite well, but with 4 links it seemed to really prefer 2 in my case. > > > Just for the record, did you also change the LACP policies on the switches? > > From what I gather, having fancy pants L3+4 hashing on the Linux side will not > fix imbalances by itself, the switches need to be configured likewise.
Yes - I was changing policies on both sides in similar manners but it seemed to be that the way the OSDs selected their service ports just happened to hash consistently to the same links. There just wasn't enough variation in the combinations of L3+L4 or even L2 hash output to utilise more of the links (the even numbered ports and consistent IP pairs just kept returning the same link output for the hash algorithm). Some of the more simplistic round robin methods might have got better results but I didn't want to stick with for future scalability. In a larger scale deployment with more clients or a wider pool of OSDs that would probably not be the case as there would be greater distribution of hash inputs. Just something to be aware of when you look to do LACP with more than 2 links. > > Christian > > > > > -----Original Message----- > > > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On > > > Behalf Of David Riedl > > > Sent: Thursday, 2 June 2016 2:12 AM > > > To: ceph-users@lists.ceph.com > > > Subject: Re: [ceph-users] Best Network Switches for Redundancy > > > > > > > > > > 4. As Ceph has lots of connections on lots of IP's and port's, > > > > LACP or the Linux ALB mode should work really well to balance > connections. > > > Linux ALB Mode looks promising. Does that work with two switches? > > > Each server has 4 ports which are 'splitted' and connected to each switch. > > > _ > > > / _[switch] > > > / / || > > > [server] || > > > \ \_ || > > > \__[switch] > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@lists.ceph.com > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > Confidentiality: This email and any attachments are confidential and > > may be subject to copyright, legal or some other professional privilege. > > They are intended solely for the attention and use of the named > > addressee(s). They may only be copied, distributed or disclosed with > > the consent of the copyright owner. If you have received this email by > > mistake or by breach of the confidentiality clause, please notify the > > sender immediately by return email and delete or destroy all copies of > > the email. Any confidentiality, privilege or copyright is not waived > > or lost because this email has been sent to you by mistake. > > _______________________________________________ ceph-users > mailing > > list ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > -- > Christian Balzer Network/Systems Engineer > ch...@gol.com Global OnLine Japan/Rakuten Communications > http://www.gol.com/ Confidentiality: This email and any attachments are confidential and may be subject to copyright, legal or some other professional privilege. They are intended solely for the attention and use of the named addressee(s). They may only be copied, distributed or disclosed with the consent of the copyright owner. If you have received this email by mistake or by breach of the confidentiality clause, please notify the sender immediately by return email and delete or destroy all copies of the email. Any confidentiality, privilege or copyright is not waived or lost because this email has been sent to you by mistake. _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
