Hello list, For some time I've been writing a custom packet generator coupled with a packet receiver using DPDK.
It works greatly when used for generating millions of packets, but unexpectedly It has troubles generating a several dozens of packets or so. The application consists of two threads, first one sends the packets from one port and second receives them on another. These are 2 ports of four-ports 1Gb NIC. It's detected as Intel Corporation 82576 Gigabit Network Connection (rev 01). Ports are connected with a patch cord. My experiment runs like the following: After initializing NICs both generator and receiver wait for 1 second. Generator sends N packets by calling rte_eth_tx_burst for each individual packet. It waits for 100000000 cpu ticks between bursts. rte_eth_tx_burst reports all packets are sent. Receiver repeats calling rte_eth_rx_burst and waits for 50000000 ticks if the function returns zero. After generators sends all the packets, the receiver continues polling for several seconds. I'm observing the following behavior: If N is small, say 20, than no packets a received. See the logs. Receiver prints Z when it got zero packets and R if it got at least one packet. Starting experiment receiver 0 started main loop generator 0 sitting on socket 0 is waiting for experiment start generator 0 started main loop Zgenerating mbuf for file 0, port 0, addr 0 on socket 0 free_count on pool 0 = 100000000 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0 sent 1 in queue 0 freed 0 Stopping after reaching 20 packets limit ZZwaiting for receivers to stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ********* Statistics: Seconds elapsed: 6.123671 If I made N bigger, say 25, than all of a sudden some packets a received, like so: Starting experiment receiver 0 started main loop generator 0 sitting on socket 0 is waiting for experiment start generator 0 started main loop generating mbuf for file 0, port 0, addr 0 on socket 0 Zfree_count on pool 0 = 100000000 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 20 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 21; received: 21; zero_iters: 41 Zgenerating mbuf for file 0, port 0, addr 21 on socket 0 free_count on pool 0 = 6113 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 22; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 22 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 23; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 23 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 24; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 24 on socket 0 sent 1 in queue 0 freed 0 Stopping after reaching 25 packets limit Rtotal received packets on queue 0: 25; received: 1; zero_iters: 1 Zwaiting for receivers to stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ********* Statistics: Seconds elapsed: 6.293315 So looks like DPDK is buffering first 21 packets and doesn't flush the buffer in the first run even if I wait for several seconds after last packet is sent. In second run we see it forwards packets one by one after first burst of size 21. I tried on DPDK-1.3.1 and DPDK-1.5.1 observing similar results. Is it expected behavior?