Hey Dumitru, thanks for the finding. I created bugzilla to track this: [1]. If we would introduce QoS support for Port Group we also need to be sure that every Group member will have separate meters.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1817606 Thanks again, Maciej On Wed, Mar 25, 2020 at 3:04 PM Dumitru Ceara <dce...@redhat.com> wrote: > On 3/24/20 2:33 PM, Maciej Jozefczyk wrote: > > Hi all! > > > > I have the following setup: > > - Logical Switch with 3 Logical Ports that are VMs: > > * 1 LSP from which i test QoS with iperf3 > > * 2 LSPs have QoS policy set. > > > > Configured QoS rules: > > > -------------------------------------------------------------------------------------------------------- > > stack@mjozefcz-devstack-qos:~$ ovn-nbctl list qos > > _uuid : 7ad43edb-ed2a-4279-8373-f925a6591508 > > action : {} > > bandwidth : {burst=10000, rate=10000} > > direction : from-lport > > external_ids : {} > > match : "inport == > \"0dbccc4f-5c36-406e-a629-70d49d52e391\"" > > priority : 2002 > > > > _uuid : 8ecac46b-1ec0-4e76-a9e0-0b3063fc79e0 > > action : {} > > bandwidth : {burst=10000, rate=10000} > > direction : from-lport > > external_ids : {} > > match : "inport == > \"cad88274-feea-4ddb-b8c1-af49ca8833cf\"" > > priority : 2002 > > stack@mjozefcz-devstack-qos:~$ > > > -------------------------------------------------------------------------------------------------------- > > Please note that the rules have the same bandwidth configuration. > > Those QoS rules are mapped for those two logical flows: > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------- > > stack@mjozefcz-devstack-qos:~$ ovn-sbctl list logical_flow > > 7ae15276-6869-40ac-be1d-b4707dcf5dc7 > > _uuid : 7ae15276-6869-40ac-be1d-b4707dcf5dc7 > > actions : "set_meter(10000, 10000); next;" > > external_ids : {source="ovn-northd.c:5451", > > stage-hint="8ecac46b", stage-name=ls_in_qos_meter} > > logical_datapath : 9a1af1f9-7b42-43c2-ab0b-f4796d209e63 > > match : "inport == > \"cad88274-feea-4ddb-b8c1-af49ca8833cf\"" > > pipeline : ingress > > priority : 2002 > > table_id : 8 > > hash : 0 > > stack@mjozefcz-devstack-qos:~$ ovn-sbctl list logical_flow > > f541520a-ef70-4038-8ee5-b5b609fc3883 > > _uuid : f541520a-ef70-4038-8ee5-b5b609fc3883 > > actions : "set_meter(10000, 10000); next;" > > external_ids : {source="ovn-northd.c:5451", > > stage-hint="7ad43edb", stage-name=ls_in_qos_meter} > > logical_datapath : 9a1af1f9-7b42-43c2-ab0b-f4796d209e63 > > match : "inport == > \"0dbccc4f-5c36-406e-a629-70d49d52e391\"" > > pipeline : ingress > > priority : 2002 > > table_id : 8 > > hash : 0 > > stack@mjozefcz-devstack-qos:~$ > > > ----------------------------------------------------------------------------------------------------------------------------------------------- > > > > The problem is that those two rules use the same meter (meter id 2): > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > stack@mjozefcz-devstack-qos:~$ sudo ovs-ofctl -O OpenFlow13 dump-flows > > br-int | grep meter > > cookie=0xf541520a, duration=4215.163s, table=16, n_packets=12497, > > n_bytes=15463221, priority=2002,reg14=0x4,metadata=0x1 > > actions=meter:2,resubmit(,17) > > cookie=0x7ae15276, duration=4215.163s, table=16, n_packets=13789, > > n_bytes=33132305, priority=2002,reg14=0x5,metadata=0x1 > > actions=meter:2,resubmit(,17) > > stack@mjozefcz-devstack-qos:~$ sudo ovs-ofctl -O OpenFlow13 dump-meters > > br-int > > OFPST_METER_CONFIG reply (OF1.3) (xid=0x2): > > meter=2 kbps burst stats bands= > > type=drop rate=10000 burst_size=10000 > > stack@mjozefcz-devstack-qos:~$ sudo ovs-ofctl -O OpenFlow13 meter-stats > > br-int > > OFPST_METER reply (OF1.3) (xid=0x2): > > meter:2 flow_count:2 packet_in_count:21607 byte_in_count:40746921 > > duration:4158.558s bands: > > 0: packet_count:2010 byte_count:6486212 > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > So if there are more than one Logical Switch Ports from the same Logical > > Switch bound on the chassis that share the same QoS BW limit settings, > > those also share the same meter. > > That ends with slitted bw limit across those Logical Switch Ports. > > I tested it using iperf3, the results: > > > > If only one LSP consumes the limit: > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > stack@mjozefcz-devstack-qos:~$ sudo ip netns exec > > ovnmeta-9a1af1f9-7b42-43c2-ab0b-f4796d209e63 iperf3 -R -O 1 -c 10.1.0.20 > > Connecting to host 10.1.0.20, port 5201 > > Reverse mode, remote host 10.1.0.20 is sending > > [ 4] local 10.1.0.2 port 57206 connected to 10.1.0.20 port 5201 > > [ ID] Interval Transfer Bandwidth > > [ 4] 0.00-1.00 sec 3.49 MBytes 29.3 Mbits/sec > > (omitted) > > [ 4] 0.00-1.00 sec 1.06 MBytes 8.86 Mbits/sec > > [ 4] 1.00-2.00 sec 1.37 MBytes 11.5 Mbits/sec > > [ 4] 2.00-3.00 sec 1.16 MBytes 9.71 Mbits/sec > > [ 4] 3.00-4.00 sec 1.17 MBytes 9.80 Mbits/sec > > [ 4] 4.00-5.00 sec 1.17 MBytes 9.84 Mbits/sec > > [ 4] 5.00-6.00 sec 1.13 MBytes 9.46 Mbits/sec > > [ 4] 6.00-7.00 sec 1.04 MBytes 8.76 Mbits/sec > > [ 4] 7.00-8.00 sec 1.26 MBytes 10.6 Mbits/sec > > [ 4] 8.00-9.00 sec 1.06 MBytes 8.88 Mbits/sec > > [ 4] 9.00-10.00 sec 1.33 MBytes 11.2 Mbits/sec > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [ 4] 0.00-10.00 sec 11.7 MBytes 9.78 Mbits/sec 2554 > > sender > > [ 4] 0.00-10.00 sec 11.7 MBytes 9.85 Mbits/sec > > receiver > > > > iperf Done. > > stack@mjozefcz-devstack-qos:~$ > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > If two VMs are using the meter - two iperf3 tests at the same time: > > stack@mjozefcz-devstack-qos:~$ sleep 1; sudo ip netns exec > > ovnmeta-9a1af1f9-7b42-43c2-ab0b-f4796d209e63 iperf3 -R -c 1 > > 0.1.0.16 > > Connecting to host 10.1.0.16, port 5201 > > Reverse mode, remote host 10.1.0.16 is sending > > [ 4] local 10.1.0.2 port 56874 connected to 10.1.0.16 port 5201 > > [ ID] Interval Transfer Bandwidth > > [ 4] 0.00-1.00 sec 3.39 MBytes 28.5 Mbits/sec > > [ 4] 1.00-2.00 sec 69.6 KBytes 570 Kbits/sec > > [ 4] 2.00-3.00 sec 456 KBytes 3.74 Mbits/sec > > [ 4] 3.00-4.00 sec 290 KBytes 2.38 Mbits/sec > > [ 4] 4.00-5.00 sec 534 KBytes 4.37 Mbits/sec > > [ 4] 5.00-6.00 sec 324 KBytes 2.65 Mbits/sec > > [ 4] 6.00-7.00 sec 16.8 KBytes 137 Kbits/sec > > [ 4] 7.00-8.00 sec 474 KBytes 3.89 Mbits/sec > > [ 4] 8.00-9.00 sec 400 KBytes 3.27 Mbits/sec > > [ 4] 9.00-10.00 sec 750 KBytes 6.15 Mbits/sec > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [ 4] 0.00-10.00 sec 6.92 MBytes 5.80 Mbits/sec 867 > sender > > [ 4] 0.00-10.00 sec 6.63 MBytes 5.56 Mbits/sec > > receiver > > > > iperf Done. > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Questions: > > - Can we create separate meter for each QoS row, to not share the same > > meter if the rules are the same (except match)? > > Hi Maciej, > > From what I see the problem is that we assign meter IDs based only on > the rate and burst specified in the "set_meter()" action [1]. > > I guess we should probably change the set_meter() action such that > ovn-northd specifies another field (maybe port or "match" or some other > unique id?) to be used when building the meter name. This should create > unique meters. > > I amm however not too familiar with the implementation details of the > QoS functionality so maybe there are other options. > > Regards, > Dumitru > > [1] https://github.com/ovn-org/ovn/blob/v20.03.0/lib/actions.c#L2786 > > > - If we would introduce QoS for Port Groups [1], can we also use > > separate meters? > > > > Versions used: > > - OVN 20.03 > > - OVS > > > > Thanks, > > Maciej > > > > [1] > https://mail.openvswitch.org/pipermail/ovs-discuss/2020-March/049864.html > > > > -- > > Best regards, > > Maciej Józefczyk > > > > _______________________________________________ > > -- Best regards, Maciej Józefczyk
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
