Hi Dave
Here is the output of "show run". I am using Trex tool and generating 1500
byte udp packets.
vpp# show errors
Count Node Reason
16 arp-input ARP replies sent
15611 VirtualFunctionEthernet0/7/0-tx Tx packet drops (dpdk tx
failure)
vpp#
vpp# show runtime
Time 30.5, average vectors/node 2.34, last 128 main loops 0.00 per node 0.00
vector rates in 3.4424e5, out 3.4373e5, drop 0.0000e0, punt 0.0000e0
Name State Calls
Vectors Suspends Clocks Vectors/Call
VirtualFunctionEthernet0/6/0-o active 1
1 0 8.02e2 1.00
VirtualFunctionEthernet0/6/0-t active 1
1 0 3.83e3 1.00
VirtualFunctionEthernet0/7/0-o active 4483078
10493156 0 7.35e1 2.34
VirtualFunctionEthernet0/7/0-t active 4483078
10477545 0 3.11e2 2.34
api-rx-from-ring any wait 0
0 1 1.02e4 0.00
arp-input active 1
2 0 2.03e4 2.00
dhcp-client-process any wait 0
0 1 3.77e3 0.00
dpdk-input polling 218410075
10493157 0 5.23e3 .05
dpdk-process any wait 0
0 10 9.76e4 0.00
ethernet-input active 1
2 0 2.86e3 2.00
fib-walk any wait 0
0 15 5.84e3 0.00
ikev2-manager-process any wait 0
0 31 2.86e3 0.00
interface-output active 1
2 0 2.51e3 2.00
ip-route-resolver-process any wait 0
0 1 3.60e3 0.00
ip4-input active 4483078
10493155 0 1.45e2 2.34
ip4-lookup active 4483078
10493155 0 9.87e1 2.34
ip4-reassembly-expire-walk any wait 0
0 3 5.86e3 0.00
ip4-rewrite active 4483078
10493155 0 8.12e1 2.34
ip6-icmp-neighbor-discovery-ev any wait 0
0 31 2.71e3 0.00
ip6-reassembly-expire-walk any wait 0
0 3 5.53e3 0.00
lisp-retry-service any wait 0
0 15 6.00e3 0.00
rd-cp-process any wait 0
0 1514830 3.02e2 0.00
unix-cli-local:6 active 0
0 97 1.59e5 0.00
unix-epoll-input polling 213083
0 0 1.11e3 0.00
vpe-oam-process any wait 0
0 15 3.99e3 0.00
vpp#
--
Santosh
On Fri, Oct 5, 2018 at 7:29 AM Jim Thompson <[email protected]> wrote:
>
> We’ve seen 4.6Gbps IPsec on a single core of a C5 instance, and that’s
> throttled by the 5gbps in and out of the VPC “limit” courtesy of Amazon.
> (The other 400Mbps is framing overhead.)
> But here, obviously, “large packets” (we were using iperf3 on the tunnel).
>
> Jim
>
> On Oct 5, 2018, at 6:38 AM, Dave Barach via Lists.Fd.Io <
> [email protected]> wrote:
>
> I’ll let others jump in on the tx [and rx] drops. At some level, it looks
> the vfio device simply isn’t disposing of traffic rapidly enough. Perhaps
> too few buffers have been configured, but I don’t think that’s the most
> likely explanation.
>
> The “show run” stats are not helpful because they include over 2000
> seconds’s worth of [mostly] dead airtime: please start traffic, “clear
> run”, wait a few seconds, then “show run”.
>
> You didn’t mention the packet size that you’re using. It wouldn’t shock me
> to learn that a “10xGE capable instance” comes with a few stars in the
> margin: perhaps “10xGE capable at or above 256 byte packets” or some such.
>
>
> HTH... Dave
>
> *From:* [email protected] <[email protected]> *On Behalf Of *Santosh
> Kumar Dornal
> *Sent:* Thursday, October 4, 2018 9:40 PM
> *To:* [email protected]
> *Subject:* [vpp-dev] Tx packet drops - dpdk tx failure
>
> Hi
>
> I am using VPP 18.04 on AWS (c5.xlarge instance with ena interfaces) and
> running into Tx packet drops when pushing traffic over 2.5Gbps
> (bidirectional). The instance is capable of doing 10gig. Can somebody help?
>
> vpp# show version
> vpp v18.04-16~gca7a68e0 built by vagrant on debian9.localdomain at Wed
> Oct 3 11:57:44 PDT 2018
> vpp# show errors
> Count Node Reason
> 68 arp-input ARP replies sent
> 82693102 VirtualFunctionEthernet0/6/0-tx Tx packet drops (dpdk tx
> failure)
> 83200033 VirtualFunctionEthernet0/7/0-tx Tx packet drops (dpdk tx
> failure)
> vpp#
>
> vpp# show hardware-interfaces detail
> Name Idx Link Hardware
> VirtualFunctionEthernet0/6/0 1 up VirtualFunctionEthernet0/6/0
> Ethernet address 02:d2:6c:6e:b0:1c
> AWS ENA VF
> carrier up full duplex speed 10000 mtu 9216
> pci id: device 1d0f:ec20 subsystem 0000:0000
> pci address: 0000:00:06.00
> max rx packet len: 9216
> max num of queues: rx 8 tx 8
> promiscuous: unicast off all-multicast off
> vlan offload: strip off filter off qinq off
> rx offload caps:
> tx offload caps: ipv4-cksum udp-cksum tcp-cksum
> rss active: ipv4-sctp ipv4-other ipv6-udp-ex ipv6-frag ipv6-tcp
> l2-payload
> rss supported: none
> rx queues 1, rx desc 1024, tx queues 1, tx desc 1024
> cpu socket 0
>
> tx frames ok 379327687
> tx bytes ok 573877720248
> rx frames ok 481696392
> rx bytes ok 729288200634
> rx missed 576271
> extended stats:
> rx good packets 481696392
> tx good packets 379327687
> rx good bytes 729288200634
> tx good bytes 573877720248
> rx missed errors 576271
> rx errors 0
> tx errors 0
> rx mbuf allocation errors 0
> rx q0packets 0
> rx q0bytes 0
> rx q0errors 0
> tx q0packets 0
> tx q0bytes 0
> VirtualFunctionEthernet0/7/0 2 up VirtualFunctionEthernet0/7/0
> Ethernet address 02:cb:fc:e1:b2:e6
> AWS ENA VF
> carrier up full duplex speed 10000 mtu 9216
> pci id: device 1d0f:ec20 subsystem 0000:0000
> pci address: 0000:00:07.00
> max rx packet len: 9216
> max num of queues: rx 8 tx 8
> promiscuous: unicast off all-multicast off
> vlan offload: strip off filter off qinq off
> rx offload caps:
> tx offload caps: ipv4-cksum udp-cksum tcp-cksum
> rss active: ipv4-sctp ipv4-other ipv6-udp-ex ipv6-frag ipv6-tcp
> l2-payload
> rss supported: none
> rx queues 1, rx desc 1024, tx queues 1, tx desc 1024
> cpu socket 0
>
> tx frames ok 378158827
> tx bytes ok 572532333080
> rx frames ok 483669396
> rx bytes ok 731851064762
> rx missed 576803
> extended stats:
> rx good packets 483669396
> tx good packets 378158827
> rx good bytes 731851064762
> tx good bytes 572532333080
> rx missed errors 576803
> rx errors 0
> tx errors 0
> rx mbuf allocation errors 0
> rx q0packets 0
> rx q0bytes 0
> rx q0errors 0
> tx q0packets 0
> tx q0bytes 0
> local0 0 down local0
> local
> vpp#
>
>
> vpp# show interface
> Name Idx State Counter
> Count
> VirtualFunctionEthernet0/6/0 1 up rx
> packets 481696393
> rx bytes
> 729288200676
> tx
> packets 483669396
> tx bytes
> 731851064730
>
> drops 4
>
> ip4 481696346
>
> rx-miss 576271
>
> tx-error 104341708
> VirtualFunctionEthernet0/7/0 2 up rx
> packets 483669397
> rx bytes
> 731851064804
> tx
> packets 481696392
> tx bytes
> 729288200602
>
> drops 4
>
> ip4 483669350
>
> rx-miss 576803
>
> tx-error 103537564
> local0 0 down
>
>
> vpp# show runtime
> Time 2390.3, average vectors/node 2.46, last 128 main loops 0.00 per node
> 0.00
> vector rates in 3.9979e5, out 3.1282e5, drop 2.5101e-3, punt 0.0000e0
> Name State Calls
> Vectors Suspends Clocks Vectors/Call
> VirtualFunctionEthernet0/6/0-o active 203354869
> 478794413 0 7.81e1 2.35
> VirtualFunctionEthernet0/6/0-t active 203354869
> 374452705 0 4.01e2 1.84
> VirtualFunctionEthernet0/7/0-o active 203863473
> 476819589 0 7.60e1 2.34
> VirtualFunctionEthernet0/7/0-t active 203863473
> 373282025 0 3.89e2 1.83
> api-rx-from-ring any wait 0
> 0 120 1.19e4 0.00
> arp-input active 61
> 88 0 2.75e4 1.44
> dhcp-client-process any wait 0
> 0 24 5.04e3 0.00
> dns-resolver-process any wait 0
> 0 2 4.74e3 0.00
> dpdk-input polling 16754698175
> 955614004 0 4.47e3 .06
> dpdk-process any wait 0
> 0 797 1.01e5 0.00
> error-drop active 6
> 6 0 3.91e3 1.00
> ethernet-input active 61
> 88 0 3.33e3 1.44
> fib-walk any wait 0
> 0 1195 4.96e3 0.00
> ikev2-manager-process any wait 0
> 0 2390 2.49e3 0.00
> interface-output active 57
> 84 0 2.79e3 1.47
> ip-route-resolver-process any wait 0
> 0 24 3.33e3 0.00
> ip4-glean active 2
> 2 0 2.06e4 1.00
> ip4-input active 382393615
> 955613916 0 1.37e2 2.49
> ip4-lookup active 382393615
> 955613916 0 9.37e1 2.49
> ip4-reassembly-expire-walk any wait 0
> 0 239 5.99e3 0.00
> ip4-rewrite active 382393613
> 955613914 0 9.20e1 2.49
> ip6-icmp-neighbor-discovery-ev any wait 0
> 0 2390 2.26e3 0.00
> ip6-reassembly-expire-walk any wait 0
> 0 239 5.69e3 0.00
> lisp-retry-service any wait 0
> 0 1195 5.34e3 0.00
> rd-cp-process any wait 0
> 0 118658595 3.15e2 0.00
> unix-cli-local:2 active 2
> 0 1363 3.00e9 0.00
> unix-epoll-input polling 16346047
> 0 0 1.07e3 0.00
> vpe-oam-process any wait 0
> 0 1171 3.69e3 0.00
> vpp#
>
> --
> Santosh
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#10759): https://lists.fd.io/g/vpp-dev/message/10759
> Mute This Topic: https://lists.fd.io/mt/26799215/675164
> Group Owner: [email protected]
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#10762): https://lists.fd.io/g/vpp-dev/message/10762
Mute This Topic: https://lists.fd.io/mt/26799215/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-