On Wed, Aug 16, 2017 at 7:59 AM, Simon Horman <simon.hor...@netronome.com> wrote: > Hi Andy, > > On Tue, Aug 15, 2017 at 01:31:36PM -0700, Andy Zhou wrote: >> On Tue, Aug 15, 2017 at 9:13 AM, Simon Horman >> <simon.hor...@netronome.com> wrote: >> > From: Pieter Jansen van Vuuren <pieter.jansenvanvuu...@netronome.com> >> > >> > Adds a config parameter that determines if a bond will use recirculation >> > in the kernel datapath when implementing a bond in balance-tcp mode. >> >> Using recirc for bonding is a lower level decision. I am not sure >> OVSDB is right level >> for exposing this detail. > > I do see that this configuration is arguably more detailed than existing > bond parameters set via OVSDB, but it is still a bond parameter so perhaps > it isn't entirely out of place here. > >> > The default for enable-recirc is "true", resulting in the traditional >> > implementation of a bond in balance-tcp mode. Setting enable-recirc to >> > false results in datapath rules that do not rely on the recirculation >> > action. >> > >> > example usage: >> > ovs-vsctl set port bond0 other_config:enable-recirc=false >> > >> > Advantages: >> > - Allows TC offloading of OVS bonds on hardware which >> > does not support recirculation >> >> Not sure it is an advantage. Bonding with recirc allows the first flow to >> be a mega flow. without megaflow, all bond traffic has to be micro >> flows, which was how OVS implements bonding. The recirc was introduced to >> solve the flow explosion problem >> >> Without supporting recirc, I'd image TC offloading would suffer the same >> issue. May be it is correct not to offload the bond flows, at least not >> before recirc is supported in TC. >> >> Another way to look at this is that when TC ran out of flow space, and >> bonding is supported by the kernel module. This configuration would >> prevent kernel module from using recirc. > > I believe that your points regarding flow explosion and fallback to > the kernel datapath are well made. However, I also believe there is > hardware where using recirculation is not practical. For that > reason I think it is worth exploring this proposal further.
I am not sure offloading microflows to hardware is a good ideal because of flow explosion. > >> > - Appears to result in lower latency (in systems using few flows). >> >> Not sure this is true in general. When using recirc with proper >> megaflow, the latency should >> improve for new microflows since upcalls can be omitted. > > That may be true in terms of flow setup cost but I believe that > the per-packet cost is going to be higher when recirculation is used. > >> > Quick ping test results in: >> > >> > other_config:enable-recirc=false >> > rtt min/avg/max/mdev = 0.039/0.193/7.612/1.059 ms >> > >> > other_config:enable-recirc=true >> > rtt min/avg/max/mdev = 0.038/0.321/14.091/1.967 ms >> > >> >> When setting up the flow, 'recirc" requires 2 miss upcalls, thus set >> up latency is larger. >> However, once flows are set up, the latency difference should be minimal. >> >> I'd expect the max latency to be different with or without recirc. >> Notice the minimal latency difference is small. >> I'd expect the average latency to converge to minimal for sufficiently >> large number of ping packets. > > Latency may not be the best way to measure this but having > to classify each packet twice, as I believe is the case when > recirculation is used, must cost something. I agree that recirculation is not free. The trade off is that it avoids flow explosion. > >> > More comprehensive testing is in progress. >> > >> > Signed-off-by: Pieter Jansen van Vuuren >> > <pieter.jansenvanvuu...@netronome.com> >> > Signed-off-by: Simon Horman <simon.hor...@netronome.com> _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
