On 08/16/2017 09:02 PM, Darrell Ball wrote: > > > -----Original Message----- > From: Kevin Traynor <[email protected]> > Organization: Red Hat > Date: Wednesday, August 16, 2017 at 12:19 PM > To: Darrell Ball <[email protected]>, "[email protected]" > <[email protected]>, "[email protected]" <[email protected]>, > "[email protected]" <[email protected]>, > "[email protected]" <[email protected]>, > "[email protected]" <[email protected]>, > "[email protected]" <[email protected]> > Subject: Re: [ovs-dev] [PATCH v4 0/6] OVS-DPDK rxq to pmd assignment > improvements. > > On 08/16/2017 07:28 AM, Darrell Ball wrote: > > Hi Kevin > > > > I did some basic testing and parsed all the code. > > I have some high level comments, below. > > I’ll add the other comments tomorrow. > > > > Thanks Darrell. > > > 1/ The redistribution is based on the last interval – I wonder if a > number of intervals may be better. > > Possibly a single interval could result in a skewed decision ? > > > > Sure, it could happen if there was some spike. Ian had made a similar > comment. The only danger is that if the interval is too long then the > older information is less relevant and also a rebalance would be > occasional. > > In the v4 I decoupled the interval used here from anything else, so it > can easily be increased if there is a better suggestion for length. > > [Darrell]: You could keep it simple for now and use a little longer time - > maybe a ‘minute’ and a future > patch could use a weighted average maybe or something else ? > This is the only major concern > I have with this series. Point 2 can come as a follow-up > series. > >
Yes, completely agree - a user command and any stats required to support it can be follow up. @Jan, can you comment on length of time to take into consideration when re-balancing - when we talked before you had some views on it. I will post a v5 when we get consensus around that. thanks, Kevin. > > Ian also suggested making it user configurable, but I'd only want to add > that if it's clear that different interval lengths are needed for > different use cases. Otherwise it's another low level config the user > has to grapple with and that has to be maintained. > > [Darrell] If we ultimately have a show/rebalance maintenance command in a > future series, a > configurable interval would be moot I think. > > > > 2/ I see a rebalance command added that can be used other than at > reconfiguration time. > > Yep, I'd really like to hear opinions about whether the command to > rebalance is useful for people in production. I prefer the case where > rebalance is done as part of a happening anyway reconfig because it > doesn't need user intervention. But you don't want someone having to add > a VM in order to do a rebalance :-) > > > Rather than for testing (per the patch use case), maybe this could > be used as maintenance command when combined > > with > https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.ozlabs.org_patch_768757_&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=kd7bxA2tqkzEpBpen6pNG2aWw61Pv0_EQtg7tddsFX8&s=QLFdqh_ZEJpnsKwnemFEqEIAbNBfXI4NgzUD1FnshEs&e= > ? > > Sugesh's patch re-implemented the data collection from this RFC, but > used the data for displaying stats. > > [Darrell] got it. > > .It's not compatible with this or > Ciara's merged patch for collecting pmd stats. > > [Darrell] agreed > > I agree that increased > stats reporting would be useful to compliment a rebalance command in > production. > > [Darrell] Anyhow, the show/rebalance can come as a follow-up series that > you could do alone or both you and Sugesh. > > > Kevin. > > > If this is the case, patch 6 could have more description how best to > do that. > > I will look at Sugesh’s patch tomorrow as well so we can expedite. > > > > Thanks Darrell > > > > > > > > > > > > -----Original Message----- > > From: <[email protected]> on behalf of Kevin Traynor > <[email protected]> > > Date: Wednesday, August 9, 2017 at 8:45 AM > > To: "[email protected]" <[email protected]>, > "[email protected]" <[email protected]>, "[email protected]" > <[email protected]>, "[email protected]" > <[email protected]>, "[email protected]" > <[email protected]>, "[email protected]" <[email protected]> > > Subject: [ovs-dev] [PATCH v4 0/6] OVS-DPDK rxq to pmd assignment > improvements. > > > > For the DPDK datapath, by default rxqs are assigned to available > pmds > > in round robin order with no weight or priority. > > > > It can happen that some very busy queues are handled by one pmd > which > > does not have enough cycles to prevent packets being dropped on > them. > > While at the same time another pmd which handles queues with no > traffic > > on them is essentially idle. > > > > Rxq to pmd assignment happens as a result of a number of events and > > when it does, the same unweighted round robin approach is applied > > each time. > > > > This patchset proposes to improve the round robin nature of rxq to > pmd > > assignment by counting the processing cycles used by the rxqs during > > their operation and incorporating that data into assignment. > > > > Before assigning in a round robin manner, the rxqs will be sorted in > > order of the processing cycles they have been consuming. Assuming > > multiple pmds, this ensures that the rxqs measured to be using the > > most processing cycles will be assigned to different cores. > > > > In some cases the measured cycles for an rxq may be not available as > > the rxq is new or may not be useful for assignment as traffic > patterns > > may change. In those cases the code will essentially fallback to > being > > round round similar to what currently exists. However, in the case > > where data is available and a reliable indication of future rxq > cycles > > consumption, rxq to pmd distribution will be much improved. > > > > V3 -> V4 > > ======== > > 4/6 > > Rebased to accomodate new cross numa assigment. > > > > V2 -> V3 > > ======== > > Dropped v2 1/7 as not reusing dpcls optimisation interval anymore > > > > 2/6 > > Moved unused functions to 3/6 to avoid compiler warning > > > > 3/6 > > Made pmd rxq interval independent from dpcls opt interval > > > > 4/6 > > Moved docs about rebalance command to when it is available in 6/6 > > Added logging info for pmd to rxq assignment > > > > 5/6 > > Added an example to docs > > > > 6/6 > > Noted in commit msg that Jan requested this for testing purposes > > > > V1 -> V2 > > ======== > > Dropped Ciara's patch to change how pmd cycles are counted as it > merged. > > > > 6/7: Rebased unit tests. > > > > > > Kevin Traynor (6): > > dpif-netdev: Change polled_queue to use dp_netdev_rxq. > > dpif-netdev: Add rxq processing cycle counters. > > dpif-netdev: Count the rxq processing cycles for an rxq. > > dpif-netdev: Change rxq_scheduling to use rxq processing cycles. > > dpif-netdev: Change pmd selection order. > > dpif-netdev: Add ovs-appctl dpif-netdev/pmd-rxq-rebalance. > > > > Documentation/howto/dpdk.rst | 26 +++++ > > lib/dpif-netdev.c | 252 > +++++++++++++++++++++++++++++++++++-------- > > tests/pmd.at | 2 +- > > vswitchd/ovs-vswitchd.8.in | 2 + > > 4 files changed, 237 insertions(+), 45 deletions(-) > > > > -- > > 1.8.3.1 > > > > _______________________________________________ > > dev mailing list > > [email protected] > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=21tyraQPA9POr0vUmVHtd5ubXxYuyyd4NUSpsiKv0jE&s=oll6IIbXYm-jrOZ36LvbIoZL1jJ_fx26ZbWdYmFprAM&e= > > > > > > > > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
