Hi All,
> -----Original Message----- > From: Jan Scheurich [mailto:jan.scheur...@ericsson.com] > Sent: Thursday, August 17, 2017 5:22 PM > To: O Mahony, Billy <billy.o.mah...@intel.com>; Kevin Traynor > <ktray...@redhat.com>; d...@openvswitch.org > Subject: RE: [ovs-dev] [PATCH 0/4] prioritizing latency sensitive traffic > > Good discussion. Some thoughts: > > 1. Prioritizing queues by assigning them to dedicated PMDs is a simple and > effective but very crude method, considering that you have to reserve an > entire (logical) core for that. So I am all for a more economic and perhaps > slightly less deterministic option! [[BO'M]] I would agree. I was just drawing attention to the two-part nature of the patch. Ie. that it's not dpdk specific as such but comes with a dpdk implementation. > > 2. Offering the option to prioritize certain queues in OVS-DPDK is a highly > desirable feature. We have at least one important use case in OpenStack > (prioritizing "in-band" infrastructure control plane traffic over tenant > data, in > case both are carried on the same physical network). In our case the traffic > separation would be done per VLAN. Can we add this to the list of supported > filters? [[BO'M]] Good to know about use-cases. I'll dig a bit on that wrt to dpdk drivers and hardware. > > 3. It would be nice to be able to combine priority queues with filters with a > number of RSS queues without filter. Is this a XL710 HW limitation or only a > limitation of the drivers and DPDK APIs? [[BO'M]] Again I'll have to dig on this. Our go to guy for this is on vacation at the moment. Remind me if I don't get back with a response. > > BR, Jan > > > > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev- > > boun...@openvswitch.org] On Behalf Of O Mahony, Billy > > Sent: Thursday, 17 August, 2017 18:07 > > To: Kevin Traynor <ktray...@redhat.com>; d...@openvswitch.org > > Subject: Re: [ovs-dev] [PATCH 0/4] prioritizing latency sensitive > > traffic > > > > Hi Kevin, > > > > Thanks for the comments - more inline. > > > > Billy. > > > > > -----Original Message----- > > > From: Kevin Traynor [mailto:ktray...@redhat.com] > > > Sent: Thursday, August 17, 2017 3:37 PM > > > To: O Mahony, Billy <billy.o.mah...@intel.com>; d...@openvswitch.org > > > Subject: Re: [ovs-dev] [PATCH 0/4] prioritizing latency sensitive > > > traffic > > > > > > Hi Billy, > > > > > > I just happened to be about to send a reply to the previous > > > patchset, so adding comments here instead. > > > > > > On 08/17/2017 03:24 PM, Billy O'Mahony wrote: > > > > Hi All, > > > > > > > > v2: Addresses various review comments; Applies cleanly on 0bedb3d6. > > > > > > > > This patch set provides a method to request ingress scheduling on > > > interfaces. > > > > It also provides an implemtation of same for DPDK physical ports. > > > > > > > > This allows specific packet types to be: > > > > * forwarded to their destination port ahead of other packets. > > > > and/or > > > > * be less likely to be dropped in an overloaded situation. > > > > > > > > It was previously discussed > > > > https://mail.openvswitch.org/pipermail/ovs-discuss/2017- > > > May/044395.htm > > > > l > > > > and RFC'd > > > > https://mail.openvswitch.org/pipermail/ovs-dev/2017-July/335237.ht > > > > ml > > > > > > > > Limitations of this patch: > > > > * The patch uses the Flow Director filter API in DPDK and has only > > > > been tested on Fortville (XL710) NIC. > > > > * Prioritization is limited to: > > > > ** eth_type > > > > ** Fully specified 5-tuple src & dst ip and port numbers for UDP & > > > > TCP packets > > > > * ovs-appctl dpif-netdev/pmd-*-show o/p should indicate rxq > > > prioritization. > > > > * any requirements for a more granular prioritization mechanism > > > > > > > > > > In general I like the idea of splitting priority traffic to a > > > specific queue but I have concerns about the implementation. I > > > shared most of these when we met already but adding here too. Not a > detailed review. > > [[BO'M]] No worries. If we get the high-level sorted out first the > > details will fall into place :) > > > > > > - It is using deprecated DPDK filter API. > > > http://dpdk.org/doc/guides/rel_notes/deprecation.html > > [[BO'M]] Yes it looks like a move to the shiny new Flow API is in order. > > > > > > - It is an invasive change that seems to be for only one Intel NIC > > > in the DPDK datapath. Even then it is very limited as it only works > > > when that Intel NIC is using exactly 2 rx queues. > > [[BO'M]] That's the current case but is really a limitation of > > FlowDirectorAPI/DPDK/XL710 combination. Maybe Flow API will allow to > > RSS over many queues and place the prioritized traffic on another queue. > > > > > > - It's a hardcoded opaque QoS which will have a negative impact on > > > whichever queues happen to land on the same pmd so it's > > > unpredictable which queues will be affected. It could effect other > > > latency sensitive traffic that cannot by prioritized because of the > limitations above. > > > > > > - I guess multiple priority queues could land on the same pmd and > > > starve each other? > > [[BO'M]] Interaction with pmd assignment is definitely an issue that > > needs to be addressed. I know there is work in-flight in that regard > > so it will be easier to address that when the in-flight work lands. > > > > > > I think a more general, less restricted scheme using DPDK rte_flow > > > API with controls on the effects to other traffic is needed. Perhaps > > > if a user is very concerned with latency on traffic from a port, > > > they would be ok with dedicating a pmd to it. > > [[BO'M]] You are proposing to prioritize queues by allocating a single > > pmd to them rather than by changing the pmds read algorithm to favor > > prioritized queues? For sure that could be another implementation of > > the solution. > > > > If we look at the patch set as containing two distinct things as per > > the cover letter "the patch set provides a method to request ingress > > scheduling on interfaces. It also provides an implementation of same > > for DPDK physical ports." Then this would change the second part put > > the first would be still valid. Each port type in any case would have > > to come up with it's own implementation - it's just for non-physical > > ports than cannot offload the prioritization decision it not worth the > > effort - as was noted in an earlier RFC. > > > > > > > > thanks, > > > Kevin. > > > > > > > Initial results: > > > > * even when userspace OVS is very much overloaded and > > > > dropping significant numbers of packets the drop rate for > > > > prioritized > > traffic > > > > is running at 1/1000th of the drop rate for non-prioritized traffic. > > > > > > > > * the latency profile of prioritized traffic through userspace OVS > > > > is also > > > much > > > > improved > > > > > > > > 1e0 |* > > > > |* > > > > 1e-1 |* | Non-prioritized pkt latency > > > > |* * Prioritized pkt latency > > > > 1e-2 |* > > > > |* > > > > 1e-3 |* | > > > > |* | > > > > 1e-4 |* | | | > > > > |* |* | | > > > > 1e-5 |* |* | | | > > > > |* |* |* | | | > > > > 1e-6 |* |* |* |* | | > > > > |* |* |* |* |* | > > > > 1e-7 |* |* |* |* |* |* > > > > |* |* |* |* |* |* > > > > 1e-8 |* |* |* |* |* |* > > > > 0-1 1-20 20-40 40-50 50-60 60-70 ... 120-400 > > > > Latency (us) > > > > > > > > Proportion of packets per latency bin @ 80% Max Throughput > > > > (Log scale) > > > > > > > > > > > > Regards, > > > > Billy. > > > > > > > > billy O'Mahony (4): > > > > netdev: Add set_ingress_sched to netdev api > > > > netdev-dpdk: Apply ingress_sched config to dpdk phy ports > > > > dpif-netdev: Add rxq prioritization > > > > docs: Document ingress scheduling feature > > > > > > > > Documentation/howto/dpdk.rst | 31 +++++++ > > > > include/openvswitch/ofp-parse.h | 3 + > > > > lib/dpif-netdev.c | 25 ++++-- > > > > lib/netdev-bsd.c | 1 + > > > > lib/netdev-dpdk.c | 192 > > > +++++++++++++++++++++++++++++++++++++++- > > > > lib/netdev-dummy.c | 1 + > > > > lib/netdev-linux.c | 1 + > > > > lib/netdev-provider.h | 10 +++ > > > > lib/netdev-vport.c | 1 + > > > > lib/netdev.c | 22 +++++ > > > > lib/netdev.h | 1 + > > > > vswitchd/bridge.c | 4 + > > > > vswitchd/vswitch.xml | 31 +++++++ > > > > 13 files changed, 315 insertions(+), 8 deletions(-) > > > > > > > > _______________________________________________ > > dev mailing list > > d...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
