Hi Pierre, Thank you for giving some helpful choices.
For newly created network, the first choice is a good one, because it is flexible and scalable. For legacy AS case, we intend to adopt the second solution. Thanks a lot, Hongjun From: Pierre Pfister (ppfister) [mailto:ppfis...@cisco.com] Sent: Tuesday, April 25, 2017 4:11 PM To: Ni, Hongjun <hongjun...@intel.com> Cc: Zhou, Danny <danny.z...@intel.com>; Ed Warnicke <hagb...@gmail.com>; Li, Johnson <johnson...@intel.com>; vpp-dev@lists.fd.io Subject: Re: [vpp-dev] Requirement on Load Balancer plugin for VPP Le 25 avr. 2017 à 09:52, Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> a écrit : Hi Pierre, For LB distribution case, I think we could assign a node IP for each LB box. When received packets from client, LB will do both SNAT and DNAT. i.e. source IP -> LB’s Node IP, destination IP -> AS’s IP. When returned packets from AS, LB also do both DNAT and SNAT. i.e. source IP -> AS’s IP, destination IP -> Client’s IP. I see. Doing so you completely hide the client's source address from the application. You also require per-connexion binding at the load balancer (MagLev does per-connexion binding, but in a way which allows for hash collisions, because it is not a big deal if two flows use the same entry in the hash table. This allows for smaller and fixed size hash table, which also provides a performance advantage to MagLev). In my humble opinion, using SNAT+DNAT is a terribly bad idea, so I would advise you to reconsider finding a way to either: - Enable any type of packet tunneling protocol in your ASs (IPinIP, L2TP, whatever-other-protocol, and extend VPP's LB plugin with the one you pick). - Put some box closer to the ASs (bump in the wire) for decap. - If your routers support MPLS, you could also use it as encap. If you really want to use SNAT+DNAT (god forbid), and are willing to suffer (or somehow like suffering), you may try to: - Use VPP's SNAT on the client-facing interface. The SNAT will just change clients source addresses to one of LB's source addresses. - Extend VPP's LB plugin to support DNAT "encap". - Extend VPP's LB plugin to support return traffic and stateless SNAT base on LB flow table (And find a way to make that work on multiple cores...). The client->AS traffic, in VPP, would do ---> client-facing-iface --> SNAT --> LB(DNAT) --> AS-facing-iface The AS->client traffic, in VPP, would do ---> AS-facing-iface --> LB(Stateless SNAT) --> SNAT Plugin (doing DNAT-back) --> client-facing-iface Now the choice is all yours. But I will have warned you. Cheers, - Pierre Thanks, Hongjun From: Pierre Pfister (ppfister) [mailto:ppfis...@cisco.com] Sent: Tuesday, April 25, 2017 3:12 PM To: Zhou, Danny <danny.z...@intel.com<mailto:danny.z...@intel.com>> Cc: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>>; Ed Warnicke <hagb...@gmail.com<mailto:hagb...@gmail.com>>; Li, Johnson <johnson...@intel.com<mailto:johnson...@intel.com>>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Requirement on Load Balancer plugin for VPP Hello all, As mentioned by Ed, introducing return traffic would dramatically reduce the performance of the solution. -> Return traffic typically consists of data packets, whereas forward traffic mostly consists of ACKs. So you will have to have significantly more LB boxes if you want to support all your return traffic. -> Having to deal with return traffic also means that we need to either make sure return traffic goes through the same core, or add locks to the structures (for now, everything is lockless, per-core), or steer traffic for core to core. There also is something that I am not sure to understand. You mentioned DNAT in order to steer the traffic to the AS, but how do you make sure the return traffic goes back to the LB ? My guess is that all the traffic coming out of the ASs is routed toward one LB, is that right ? How do you make sure the return traffic is evenly distributed between LBs ? It's a pretty interesting requirement that you have, but I am quite sure the solution will have to be quite far from MagLev's design, and probably less efficient. - Pierre Le 25 avr. 2017 à 05:11, Zhou, Danny <danny.z...@intel.com<mailto:danny.z...@intel.com>> a écrit : Share my two cents as well: Firstly, introducing GRE or whatever other tunneling protocols to LB introduces performance overhead (for encap and decap) to both the load balancer as well as the network service. Secondly, other mechanism on the network service node not only needs to decap the GRE but also needs to perform a DNAT operation in order to change the destination IP of the original frame from LB’s IP to the service entity’s IP, which introduces the complexity to the network service. Existing well-known load balancers such as Netfilter or Nginx do not adopt this tunneling approach, they just simply do a service node selection followed by a NAT operation. -Danny From: vpp-dev-boun...@lists.fd.io<mailto:vpp-dev-boun...@lists.fd.io> [mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Ni, Hongjun Sent: Tuesday, April 25, 2017 11:05 AM To: Ed Warnicke <hagb...@gmail.com<mailto:hagb...@gmail.com>> Cc: Li, Johnson <johnson...@intel.com<mailto:johnson...@intel.com>>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Requirement on Load Balancer plugin for VPP Hi Ed, Thanks for your prompt response. This item is required to handle legacy AS, because some legacy AS does not want to change their underlay forwarding infrastructure. Besides, some AS IPs are private and invisible outside the AS cluster domain, and not allowed to expose to external network. Thanks, Hongjun From: Ed Warnicke [mailto:hagb...@gmail.com] Sent: Tuesday, April 25, 2017 10:44 AM To: Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Li, Johnson <johnson...@intel.com<mailto:johnson...@intel.com>> Subject: Re: [vpp-dev] Requirement on Load Balancer plugin for VPP Hongjun, I can see this point of view, but it radically reduces the scalability of the whole system. Wouldn't it just make sense to run vpp or some other mechanism to decap the GRE on whatever is running the other AS and feed whatever we are load balancing to? Forcing back traffic through the central load balancer radically reduces scalability (which is why Maglev, which inspired what we are doing here, doesn't do it that way either). Ed On Mon, Apr 24, 2017 at 7:18 PM, Ni, Hongjun <hongjun...@intel.com<mailto:hongjun...@intel.com>> wrote: Hey, Currently, traffic received for a given VIP (or VIP prefix) is tunneled using GRE towards the different ASs in a way that (tries to) ensure that a given session will always be tunneled to the same AS. But in real environment, many Application Servers do not support GRE feature. So we raise a requirement for LB in VPP: (1). When received traffic for a VIP, the LB need to do load balance, then do DNAT to change traffic’s destination IP from VIP to AS’s IP. (2). When returned traffic from AS, the LB will do SNAT first to change traffic’s source IP from AS’s IP to VIP, then go through load balance sessions, and then sent to clients. Any comments about this requirement are welcome. Thanks a lot, Hongjun _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev _______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
