Hi Greg,
Great question, thank you! Please see my comments below.
> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Greg Smith
> Sent: Monday, April 20, 2015 7:40 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] QoS Question
>
> Hi DPDK team,
>
> The docs on QoS
> (http://dpdk.org/doc/guides/prog_guide/qos_framework.html# ) describe
> the traffic class (TC) as follows:
> 1 - The TCs of the same pipe handled in strict priority order.
> 2 - Upper limit enforced per TC at the pipe level.
> 3 - Lower priority TCs able to reuse pipe bandwidth currently unused by
> higher priority TCs.
> 4 - When subport TC is oversubscribed (configuration time event), pipe TC
> upper limit is capped to a dynamically adjusted value that is shared by all
> the
> subport pipes.
>
> Can someone describe how and when the TC upper limit is "dynamically"
> changed?
This feature is described at length in Programmer's Guide section 21.2.4.6.6.
Subport Traffic Class Oversubscription. Please note this feature is not enabled
by default in the code base. To enable it, please set flag
CONFIG_RTE_SCHED_SUBPORT_TC_OV=y in the DPDK configuration file
(config/common_linuxapp or config/common_bsdapp).
Subport traffic class oversubscription is implemented only for the lowest
priority traffic class (TC3, a.k.a. Best Effort), as usually the Best Effort
traffic class (where most of the traffic is) is oversubscribed, while the high
priority traffic classes (e.g. voice) are usually fully provisioned.
Overprovisioning takes place when the service provider is selling more
bandwidth that physically available, i.e. when the summation of the _nominal_
rate assigned to the users exceeds the rate of the subport. This does not
necessarily represent a problem, as only a fraction of the users are looking to
fully utilize their service (i.e. use up 100% of their nominal rate) at any
given time: when the current total demand from all subport users does not
exceed the subport rate, no problem exists, as each subscriber has its demand
fully serviced; when the current total demand (which changes dynamically)
exceeds the limit, it is obviously no longer possible to fully meet the entire
demand.
In the latter case, it is important that some fairness is taking place. We do
not want to have some subscribers getting close to 0% of their nominal rate,
while others getting close to 100% of their nominal rate, as this would not be
fair. On the other hand, we cannot reduce the nominal rate of the users (e.g.
everybody is now allowed 73% of their rate), as the nominal rate of a
subscriber is completely disconnected from its current demand: one user might
demand only 10% of its rate at this moment, so reserving 73% of its rate for
this users results in wasting 63% of its rate, which could otherwise be awarded
to some other user which has a higher demand at the same moment.
What we need to do is this: we need to apply a water filling algorithm that
computes a user quota (common for all the subport users), so that users with
current demand less than this quota will be fully serviced, while users with
high demand will be truncated. This user quota is updated periodically, with
the new value being estimated based on subport consumption from the past: when
we see that the previous quota resulted in some subport bandwidth not being
consumed, we increase the quota incrementally until the entire subport
bandwidth is consumed; when we see that the entire subport bandwidth is
consumed, we start dropping the quota incrementally until we see that some
subport bandwidth starts to be wasted.
>
> For example, assume there's a 1Gb/s port and a single 1Gb/s subport and
> 2000 pipes each of 1Mb/s (total pipes = 2Gb/s which is > the 1Gb/s subport
> which I think means "oversubscribed" as used in the doc). Each Pipe has a
> single TC.
Yes, agree this is an example of oversubscription. I used a similar example to
describe this feature during the DPDK community readout earlier this week
(https://youtu.be/_PPklkWGugs).
> In that case, would each pipe be shaped to an upper limit of 0.5 Mb/s?
Only in the very unlikely event that all the 2000 users are active and each one
is asking for 0.5 Mbps or more.
Typically, some of these users are currently inactive (demand = 0%) and some
others will ask for less than e.g. 0.5 Mbps; whatever subport bandwidth is left
unused by the low demand users, it can be awarded to the high demand users (of
course, no user will ever get more than its nominal rate).
Let's refine the example: let's say that, currently, the demand distribution
for the 2000 users is: [500 users: 0 Mbps; 500 users: 0.4 Mbps; 500 users: 0.7
Mbps; 500 users: 1 Mbps].
These users will be awarded the following rates: [500 users: 0 Mbps; 500 users:
0.4 Mbps; 500 users: 0.7 Mbps
