Konstantin, The recommended use of rte_eth_tx_prepare() had no effect, which after looking at it, makes sense. We are using "large" mbufs to support Jumbo frames, so nb-seg will always == 1. Additionally, we are not currently leveraging any HW offload capabilities. As such, rte_eth_tx_prepare() always returns "num-frames".
Taking this a step further, I have reproduced the problem using a simple c-unit test that builds bursts of frames, where each burst contains a max-burst of frames (32 in our application), where the interleaved frames have either a legal frame length (124-bytes) or intentionally a runt frame length (20-bytes++). These are dumb-simple L2 frames, i.e. NOT ip-frames. The NIC is setup to pad and append, so it should just do that without issue as needed. The test repeats this burst sequence a 100 times, resulting in 3200 frames attempting to be transmitted. Run to run, I am seeing anywhere from 750 to 3000 frames get transmitted. Thereafter, the NIC will no longer accept frames for transmit. Using GDB, we have confirmed the same DD status problem is present and preventing ixgbe_tx_free_bufs() from doing any actual freeing of resources. Is there a minimum runt size officially supported by DPDK and/or Intel on the x550 NIC family? We could certainly do a simple frame-length check and discard accordingly. However, we have seen 3rd party applications send us runts, e.g. 40-byte ARP requests, over vhost-user and tap interfaces, so we are a bit hesitant to blindly enforce this at 60 bytes (min ETH minus CRC). -Mike -----Original Message----- From: Bly, Mike Sent: Tuesday, July 23, 2019 10:08 AM To: Ananyev, Konstantin <konstantin.anan...@intel.com>; 'dev@dpdk.org' <dev@dpdk.org> Subject: RE: [dpdk-dev] x552 transmit issue and rte_ethtool - rte_ethtool_get_regs() Konstantin, Thank you for the prompt reply on this posting. In looking at the single use-case in test-pmd's csumonly.c, it would seem prepare + retry_enabled may have some shortcomings as currently coded when nb_prep < nb_rx. Has anyone looked at this? I happened to notice this when looking for a reference for how it is expected to be used. It would seem nb_rx should be replaced with nb_prep in the retry code. I think the rest of the code should "just work" from there. Thoughts? Regards, Mike -----Original Message----- From: Ananyev, Konstantin <konstantin.anan...@intel.com> Sent: Tuesday, July 23, 2019 9:03 AM To: Bly, Mike <m...@ciena.com>; 'dev@dpdk.org' <dev@dpdk.org> Subject: [**EXTERNAL**] RE: [dpdk-dev] x552 transmit issue and rte_ethtool - rte_ethtool_get_regs() > > Hello, > > We are chasing an interesting NIC transmit issue where after some > period of time with normal operation the NIC enters a state where it > refuses to transmit frames from our DPDK application via rte_eth_tx_burst(). > All indications are the port is up and otherwise operational and is still > receiving traffic. It simply refuses to transmit anymore. > > Our application is running DPDK 17.05.1. In digging through the email > archives, this appears to be related to the following posts, as we see the > same nb_free = 0 and IXGBE_ADVTXD_STAT_DD not set problems they describe: > http://mails.dpdk.org/archives/dev/2017-August/073240.html > http://inbox.dpdk.org/dev/b704af91-dcc6-4481-a54c-3e174b744d17.h.liu@a > libaba-inc.com/ > > Having not seen any resolution on the above DPDK posts and after a > number of other investigative steps, we incorporated the rte_ethtool > lib to provide the ability to dump the NIC register set via > rte_ethtool_get_regs() in the hopes that perhaps there would be something > there in a status register to point us in the right direction. The question > now is what is the best way to check the register contents dumped to the > binary output file this API creates, for the x552 NIC? Does anyone know if a > decoder script exists? > > Other ideas to pursue? It is hard to tell without any other information, but sometimes that happens when user tries to TX malformed packet. Might be worth to try using rte_eth_tx_prepare() inside your app. It does some sanity checks to prevent such situations, especially when RTE_LIBRTE_ETHDEV_DEBUG is on. Konstantin
