Public bug reported: ENV: stable/queens (12.0.1) centos 7
Steps: 1. create a vm with floating IP 172.18.213.143 2. create qos policy with ingress and egress bandwidth limit rule: burst 10240 rate 10240 aka, a 10mbps rule 3. run iperf3 in vm 4. iperf3 connect the vm via floating IP 172.18.213.143 iperf3 Testing: the iperf output: # iperf3 -c 172.18.213.143 -i 1 Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender [ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver iperf Done. The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb. here is the filter: ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675) match IP src 172.18.213.143/32 (success 5675 ) police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0) ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937) match IP dst 172.18.213.143/32 (success 33937 ) police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268) ================================= Solution: set the mtu to 64kb ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) match IP dst 172.18.213.143/32 (success 0 ) police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) match IP src 172.18.213.143/32 (success 0 ) police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) ================================= iperf testing: Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender [ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver iperf Done. ** Affects: neutron Importance: Undecided Assignee: LIU Yulong (dragon889) Status: New ** Description changed: ENV: stable/queens (12.0.1) centos 7 Steps: - 1. create a vm with floating IP 1.1.1.1 + 1. create a vm with floating IP 172.18.213.143 2. create qos policy with ingress and egress bandwidth limit rule: - burst 10240 rate 10240 - aka, a 10mbps rule + burst 10240 rate 10240 + aka, a 10mbps rule 3. run iperf3 in vm - + 4. iperf3 connect the vm via floating IP 172.18.213.143 iperf3 Testing: the iperf output: # iperf3 -c 172.18.213.143 -i 1 Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd - [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes - [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes - [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes - [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes - [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes - [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes - [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes - [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes - [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes - [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes + [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes + [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes + [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes + [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes + [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes + [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes + [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes + [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes + [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes + [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender [ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver iperf Done. - - - The inaccurate bandwidth is because of the inappropriate tc filter policy, - default value 2kb. + The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb. here is the filter: + ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675) - match IP src 172.18.213.143/32 (success 5675 ) - police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b + match IP src 172.18.213.143/32 (success 5675 ) + police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 - Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0) - - + Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0) + ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937) - match IP dst 172.18.213.143/32 (success 33937 ) - police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b + match IP dst 172.18.213.143/32 (success 33937 ) + police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 - Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268) - + Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268) + ================================= Solution: set the mtu to 64kb - + ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) - match IP dst 172.18.213.143/32 (success 0 ) - police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b + match IP dst 172.18.213.143/32 (success 0 ) + police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 - Sent 0 bytes 0 pkts (dropped 0, overlimits 0) - + Sent 0 bytes 0 pkts (dropped 0, overlimits 0) + ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) - match IP src 172.18.213.143/32 (success 0 ) - police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b + match IP src 172.18.213.143/32 (success 0 ) + police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 - Sent 0 bytes 0 pkts (dropped 0, overlimits 0) + Sent 0 bytes 0 pkts (dropped 0, overlimits 0) + ================================= + iperf testing: Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd - [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes - [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes - [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes - [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes - [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes - [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes - [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes - [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes - [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes - [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes + [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes + [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes + [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes + [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes + [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes + [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes + [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes + [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes + [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes + [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender [ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver iperf Done. ** Changed in: neutron Assignee: (unassigned) => LIU Yulong (dragon889) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1777598 Title: Inaccurate L3 QoS bandwidth Status in neutron: New Bug description: ENV: stable/queens (12.0.1) centos 7 Steps: 1. create a vm with floating IP 172.18.213.143 2. create qos policy with ingress and egress bandwidth limit rule: burst 10240 rate 10240 aka, a 10mbps rule 3. run iperf3 in vm 4. iperf3 connect the vm via floating IP 172.18.213.143 iperf3 Testing: the iperf output: # iperf3 -c 172.18.213.143 -i 1 Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 36980 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 611 KBytes 5.00 Mbits/sec 54 2.83 KBytes [ 4] 1.00-2.00 sec 752 KBytes 6.16 Mbits/sec 53 4.24 KBytes [ 4] 2.00-3.00 sec 898 KBytes 7.35 Mbits/sec 57 4.24 KBytes [ 4] 3.00-4.00 sec 604 KBytes 4.95 Mbits/sec 49 5.66 KBytes [ 4] 4.00-5.00 sec 236 KBytes 1.93 Mbits/sec 23 9.90 KBytes [ 4] 5.00-6.00 sec 810 KBytes 6.64 Mbits/sec 154 4.24 KBytes [ 4] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 27 5.66 KBytes [ 4] 7.00-8.00 sec 509 KBytes 4.17 Mbits/sec 42 2.83 KBytes [ 4] 8.00-9.00 sec 636 KBytes 5.21 Mbits/sec 48 4.24 KBytes [ 4] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 20 5.66 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 5.19 MBytes 4.35 Mbits/sec 527 sender [ 4] 0.00-10.00 sec 4.90 MBytes 4.11 Mbits/sec receiver iperf Done. The inaccurate bandwidth is because of the inappropriate tc filter mtu, default value is 2kb. here is the filter: ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 5675 success 5675) match IP src 172.18.213.143/32 (success 5675 ) police 0xc rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 Sent 404666 bytes 5675 pkts (dropped 0, overlimits 0) ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 33937 success 33937) match IP dst 172.18.213.143/32 (success 33937 ) police 0xb rate 10240Kbit burst 1280Kb mtu 2Kb action drop overhead 0b ref 1 bind 1 Sent 51266073 bytes 34679 pkts (dropped 8268, overlimits 8268) ================================= Solution: set the mtu to 64kb ================================= filter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) match IP dst 172.18.213.143/32 (success 0 ) police 0xf rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) ================================= filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1 (rule hit 0 success 0) match IP src 172.18.213.143/32 (success 0 ) police 0x10 rate 10240Kbit burst 1280Kb mtu 64Kb action drop overhead 0b ref 1 bind 1 Sent 0 bytes 0 pkts (dropped 0, overlimits 0) ================================= iperf testing: Connecting to host 172.18.213.143, port 5201 [ 4] local 172.18.211.30 port 37886 connected to 172.18.213.143 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 2.49 MBytes 20.8 Mbits/sec 146 28.3 KBytes [ 4] 1.00-2.00 sec 1.30 MBytes 10.9 Mbits/sec 119 7.07 KBytes [ 4] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 77 18.4 KBytes [ 4] 3.00-4.00 sec 1.12 MBytes 9.38 Mbits/sec 72 19.8 KBytes [ 4] 4.00-5.00 sec 1.12 MBytes 9.38 Mbits/sec 77 33.9 KBytes [ 4] 5.00-6.00 sec 1.12 MBytes 9.38 Mbits/sec 100 39.6 KBytes [ 4] 6.00-7.00 sec 1.18 MBytes 9.90 Mbits/sec 145 42.4 KBytes [ 4] 7.00-8.00 sec 1.24 MBytes 10.4 Mbits/sec 141 32.5 KBytes [ 4] 8.00-9.00 sec 1.43 MBytes 12.0 Mbits/sec 126 42.4 KBytes [ 4] 9.00-10.00 sec 1.12 MBytes 9.38 Mbits/sec 80 24.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 13.4 MBytes 11.2 Mbits/sec 1083 sender [ 4] 0.00-10.00 sec 13.0 MBytes 10.9 Mbits/sec receiver iperf Done. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1777598/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp