Re: [vpp-dev] AVF TX bandwidth less than RX
From: Damjan Marion (damarion) Sent: 2020年12月17日 23:44 To: Lijian Zhang Cc: vpp-dev ; nd Subject: Re: [vpp-dev] AVF TX bandwidth less than RX On 17.12.2020., at 14:43, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion via lists.fd.io<http://lists.fd.io/> Sent: 2020年12月17日 16:22 To: Lijian Zhang mailto:lijian.zh...@arm.com>> Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd mailto:n...@arm.com>> Subject: Re: [vpp-dev] AVF TX bandwidth less than RX On 17.12.2020., at 09:16, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: Hi Damjan, When I was benchmarking AVF PMD driver with L3fwd and L2-xconnect cases, not all packets received via AVF input node ‘avf-input’ are transmitted out via AVF output node ‘eth1-tx’. I have tried to increase the tx-queue-size of avf interface, but it does not help. Have checked VNET_DEVICE_CLASS_TX_FN (avf_device_class) (), but didn’t find software bottle-neck there. Is this a normal behavior for XL710 VF interface, and does it mean VF PCIe DMA bandwidth in RX direction is higher than that in TX direction? --L3fwd--- avf-inputpolling 2369637 606626968 0 1.30e1 255.99 eth1-output active2369637 606626968 0 5.56e0 255.99 eth1-tx active2369637 442718040 0 2.02e1 186.83 ethernet-input active2369637 606626968 0 1.26e1 255.99 ip4-input-no-checksumactive2369637 606626968 0 1.91e1 255.99 ip4-lookup active2369637 606626968 0 2.06e1 255.99 ip4-rewrite active2369637 606626968 0 1.85e1 255.99 --L2-xconnect --- avf-inputpolling 7342184 779122436 0 2.39e1 106.12 eth1-output active7342123 779122436 0 6.67e0 106.12 eth1-tx active7342123 429393137 0 3.23e1 58.48 ethernet-input active7342123 779122436 0 1.37e1 106.12 l2-input active7342123 779122436 0 1.14e1 106.12 l2-outputactive7342123 779122436 0 6.19e0 106.12 What "show error" says? If VPP was not able to enqueue packet to tx queue, it will bunp ‘no free slots’ counter. [Lijian] Yes, it reports “no free tx slots” vpp# clear errors vpp# show errors Count Node Reason Severity 2539792 eth1-txno free tx slots error vpp# show errors Count Node Reason Severity 5547120 eth1-txno free tx slots error Yes, so looks like software is faster than hardware…. What is your Mpps rate? [Lijian] 20.18Mpps for “l3fwd xl710 on x86 8268 turbo-off 2.9GHz”, and 20.10Mpps for “l2-xconnect xl710 on x86 8268 turbo-off 2.9GHz” l3fwd xl710 on x86 8268 turbo-off --- // 20.18Mpps avf-inputpolling557467 142711552 0 1.29e1 256.00 eth1-output active 557467 142711552 0 6.99e0 256.00 eth1-tx active 557467 124816512 0 1.95e1 223.89 ethernet-input active 557467 142711552 0 1.53e1 256.00 ip4-input-no-checksumactive 557467 142711552 0 2.41e1 256.00 ip4-lookup active 557467 142711552 0 2.55e1 256.00 ip4-rewrite active 557467 142711552 0 2.29e1 256.00 cross-connect xl710 on x86 8268 turbo-off --- // 20.10Mpps avf-inputpolling 1092996 160940736 0 2.00e1 147.25 eth1-output active1092996 160940736 0 7.46e0 147.25
Re: [vpp-dev] AVF TX bandwidth less than RX
On 17.12.2020., at 14:43, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion via lists.fd.io<http://lists.fd.io/> Sent: 2020年12月17日 16:22 To: Lijian Zhang mailto:lijian.zh...@arm.com>> Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>; nd mailto:n...@arm.com>> Subject: Re: [vpp-dev] AVF TX bandwidth less than RX On 17.12.2020., at 09:16, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: Hi Damjan, When I was benchmarking AVF PMD driver with L3fwd and L2-xconnect cases, not all packets received via AVF input node ‘avf-input’ are transmitted out via AVF output node ‘eth1-tx’. I have tried to increase the tx-queue-size of avf interface, but it does not help. Have checked VNET_DEVICE_CLASS_TX_FN (avf_device_class) (), but didn’t find software bottle-neck there. Is this a normal behavior for XL710 VF interface, and does it mean VF PCIe DMA bandwidth in RX direction is higher than that in TX direction? --L3fwd--- avf-inputpolling 2369637 606626968 0 1.30e1 255.99 eth1-output active2369637 606626968 0 5.56e0 255.99 eth1-tx active2369637 442718040 0 2.02e1 186.83 ethernet-input active2369637 606626968 0 1.26e1 255.99 ip4-input-no-checksumactive2369637 606626968 0 1.91e1 255.99 ip4-lookup active2369637 606626968 0 2.06e1 255.99 ip4-rewrite active2369637 606626968 0 1.85e1 255.99 --L2-xconnect --- avf-inputpolling 7342184 779122436 0 2.39e1 106.12 eth1-output active7342123 779122436 0 6.67e0 106.12 eth1-tx active7342123 429393137 0 3.23e1 58.48 ethernet-input active7342123 779122436 0 1.37e1 106.12 l2-input active7342123 779122436 0 1.14e1 106.12 l2-outputactive7342123 779122436 0 6.19e0 106.12 What "show error" says? If VPP was not able to enqueue packet to tx queue, it will bunp ‘no free slots’ counter. [Lijian] Yes, it reports “no free tx slots” vpp# clear errors vpp# show errors Count Node Reason Severity 2539792 eth1-txno free tx slots error vpp# show errors Count Node Reason Severity 5547120 eth1-txno free tx slots error Yes, so looks like software is faster than hardware…. What is your Mpps rate? — Damjan -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18387): https://lists.fd.io/g/vpp-dev/message/18387 Mute This Topic: https://lists.fd.io/mt/79032447/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] AVF TX bandwidth less than RX
From: vpp-dev@lists.fd.io On Behalf Of Damjan Marion via lists.fd.io Sent: 2020年12月17日 16:22 To: Lijian Zhang Cc: vpp-dev ; nd Subject: Re: [vpp-dev] AVF TX bandwidth less than RX On 17.12.2020., at 09:16, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: Hi Damjan, When I was benchmarking AVF PMD driver with L3fwd and L2-xconnect cases, not all packets received via AVF input node ‘avf-input’ are transmitted out via AVF output node ‘eth1-tx’. I have tried to increase the tx-queue-size of avf interface, but it does not help. Have checked VNET_DEVICE_CLASS_TX_FN (avf_device_class) (), but didn’t find software bottle-neck there. Is this a normal behavior for XL710 VF interface, and does it mean VF PCIe DMA bandwidth in RX direction is higher than that in TX direction? --L3fwd--- avf-inputpolling 2369637 606626968 0 1.30e1 255.99 eth1-output active2369637 606626968 0 5.56e0 255.99 eth1-tx active2369637 442718040 0 2.02e1 186.83 ethernet-input active2369637 606626968 0 1.26e1 255.99 ip4-input-no-checksumactive2369637 606626968 0 1.91e1 255.99 ip4-lookup active2369637 606626968 0 2.06e1 255.99 ip4-rewrite active2369637 606626968 0 1.85e1 255.99 --L2-xconnect --- avf-inputpolling 7342184 779122436 0 2.39e1 106.12 eth1-output active7342123 779122436 0 6.67e0 106.12 eth1-tx active7342123 429393137 0 3.23e1 58.48 ethernet-input active7342123 779122436 0 1.37e1 106.12 l2-input active7342123 779122436 0 1.14e1 106.12 l2-outputactive7342123 779122436 0 6.19e0 106.12 What "show error" says? If VPP was not able to enqueue packet to tx queue, it will bunp ‘no free slots’ counter. [Lijian] Yes, it reports “no free tx slots” vpp# clear errors vpp# show errors Count Node Reason Severity 2539792 eth1-txno free tx slots error vpp# show errors Count Node Reason Severity 5547120 eth1-txno free tx slots error — Damjan -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18386): https://lists.fd.io/g/vpp-dev/message/18386 Mute This Topic: https://lists.fd.io/mt/79032447/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] AVF TX bandwidth less than RX
On 17.12.2020., at 09:16, Lijian Zhang mailto:lijian.zh...@arm.com>> wrote: Hi Damjan, When I was benchmarking AVF PMD driver with L3fwd and L2-xconnect cases, not all packets received via AVF input node ‘avf-input’ are transmitted out via AVF output node ‘eth1-tx’. I have tried to increase the tx-queue-size of avf interface, but it does not help. Have checked VNET_DEVICE_CLASS_TX_FN (avf_device_class) (), but didn’t find software bottle-neck there. Is this a normal behavior for XL710 VF interface, and does it mean VF PCIe DMA bandwidth in RX direction is higher than that in TX direction? --L3fwd--- avf-inputpolling 2369637 606626968 0 1.30e1 255.99 eth1-output active2369637 606626968 0 5.56e0 255.99 eth1-tx active2369637 442718040 0 2.02e1 186.83 ethernet-input active2369637 606626968 0 1.26e1 255.99 ip4-input-no-checksumactive2369637 606626968 0 1.91e1 255.99 ip4-lookup active2369637 606626968 0 2.06e1 255.99 ip4-rewrite active2369637 606626968 0 1.85e1 255.99 --L2-xconnect --- avf-inputpolling 7342184 779122436 0 2.39e1 106.12 eth1-output active7342123 779122436 0 6.67e0 106.12 eth1-tx active7342123 429393137 0 3.23e1 58.48 ethernet-input active7342123 779122436 0 1.37e1 106.12 l2-input active7342123 779122436 0 1.14e1 106.12 l2-outputactive7342123 779122436 0 6.19e0 106.12 What "show error" says? If VPP was not able to enqueue packet to tx queue, it will bunp ‘no free slots’ counter. — Damjan -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18385): https://lists.fd.io/g/vpp-dev/message/18385 Mute This Topic: https://lists.fd.io/mt/79032447/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
