Hello Folks, currently I'm struggling while creating an application for a custom STM32F429ZG based custom board using LwIP. I need to achieve a throughput of at least 2 MBit/s, according our requirements. Due to the HW this should be possible.
The application is based on the STMCubeMX V4.3.0 and FW Library 1.3.0. However I've written a zero-copy EthernetIF driver in RX&TX. For the application, the MCU based device acts as a TCP/IP server with our own application protocol, implemented using the RAW TCP API. The implementation is inspired by the HTTP Server example. Also the guidelines regarding throughput from the LwIP wiki has been followed. The FW Library bug in the Ethernet IRQ, eating fast packets is fixed. I'm using custom memory pools. there is most likely no memory issue. Also all the priorities are ok. To achieve maximum throughput, I have done some further, potential dangerous changes: - The MSS, TCP_SND_BUF, pbuf len, etc is increased from u16_t to u32_t. The MTU is still 1500. The problem with u16_t is the limitation the send buffer to a maximum of 0xFFFE bytes. For some checks in LwIP 0xFFFF is interpreted as 0x0000, but is need to have segments of 0xFFFF bytes for a fast processing of SD Card pages. - I decreased the TCP timer intervals from 250 ms to 10 ms. A even higher rate tends to produce a lot of retransmissions. Currently I can achieve almost the desired speed for one file transfer. However, at the end of the transfer, there is one strange bug. I couldn't find a similar case in the forums, so far. Before the last frame of the last segment is sent, LwIP starts to retransmit a bunch of TCP packets. The actual number is variable, but usually somewhere between 10 and 25 frames. Interestingly there is always this pattern: MCU -> PC Packet 1460 Bytes [ACK] MCU -> PC Packet 1460 Bytes [ACK] PC -> MCU ACK on both, 0 Bytes MCU -> PC Packet 1460 Bytes [ACK] MCU -> PC Packet 1460 Bytes [ACK] PC -> MCU ACK on both, 0 Bytes ... MCU -> PC Packet 1460 Bytes [ACK] MCU -> PC Packet 1460 Bytes [ACK, PSH] PC -> MCU ACK on both, 0 Bytes ............ Then the following occurs on the last segment: (for this example, the Seq is relative to the begin of the segment) MCU -> PC Packet 1460 Bytes [ACK] [Seq: 0000] MCU -> PC Packet 1460 Bytes [ACK] [Seq: 1460] PC -> MCU ACK on both, 0 Bytes [Ack: 2920] MCU -> PC Packet 1460 Bytes [ACK] [Seq: 2920] MCU -> PC Packet 1460 Bytes [ACK] [Seq: 4380] PC -> MCU ACK on both, 0 Bytes [Ack: 4380] ... MCU -> PC Packet 1460 Bytes [TCP Retransmission] [ACK] [Seq: 0000] PC -> MCU 0 Bytes [TCP DUP ACK] [Ack: 4380] MCU -> PC Packet 1460 Bytes [TCP Retransmission] [ACK] [Seq: 2920] (Packet with Seq 1460 is NOT retransmitted!) PC -> MCU 0 Bytes [TCP DUP ACK] [Ack: 4380] MCU -> PC Packet 1460 Bytes [TCP Retransmission] [ACK] [Seq: 4380] PC -> MCU 0 Bytes [TCP DUP ACK] [Ack: 4380] ... MCU -> PC Packet 1198 Bytes [ACK, PSH] [Seq: 65700] PC -> MCU ACK on last one, sending new command, 38 Bytes [ACK, PSH] [Seq: 66898] MCU -> PC Packet 1198 Bytes [TCP Retransmission] [ACK, PSH] [Seq: 65700] PC -> MCU 0 Bytes [TCP DUP ACK #1] [ACK, PSH] [Seq: 66898] MCU -> PC Packet 1198 Bytes [TCP Retransmission] [ACK, PSH] [Seq: 65700] PC -> MCU0 Bytes [TCP DUP ACK #2] [ACK, PSH] [Seq: 66898] MCU -> PC Packet 1198 Bytes [TCP Retransmission] [ACK, PSH] [Seq: 65700] PC -> MCU 0 Bytes [TCP DUP ACK #3] [ACK, PSH] [Seq: 66898] PC -> MCU ACK on last one, sending new command, 38 Bytes [TCP Retransmission] [ACK, PSH] [Seq: 66898] ... Afterwards, nearly every packet is transmitted at least twice in both directions. Effectively no data transfer possible anymore. In the pcap file, 192.168.111.200 is MCU and .63 the PC. The interesting part is starting with frame 2056, the first retransmitted frame. 2057 is not retransmitted - that means frame 2058 must have been processed by LwIP - right? The data is passed to LwIP by tcp_write(). One segment consists of two writes: tcp_write(pcb, [ptr to 8 bytes of header], 8, TCP_WRITE_FLAG_COPY | TCP_WRITE_FLAG_MORE) tcp_write(pcb, [ptr to up to 65535 bytes of data], len, 0) // the data is stored in elements of the memory pool and freed by the tcp_sent callback According to lwip_stats there is no memory leak and no packet drop. Has someone seen something similar? Any clue? There are some other scenarios, leading to similar behaviors. SD_download_breaks.pcap <http://lwip.100.n7.nabble.com/file/n23275/SD_download_breaks.pcap> -- View this message in context: http://lwip.100.n7.nabble.com/TCP-retransmission-flooding-at-end-of-stream-tp23275.html Sent from the lwip-users mailing list archive at Nabble.com. _______________________________________________ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users