casaroli opened a new pull request, #22089:
URL: https://github.com/apache/nuttx/pull/22089

   ## Summary
   
   Fixes #22088.
   
   Since ea5d2c9ae49 (#22000), `tcp_datahandler()` drops the received data 
while fewer than `IOB_THROTTLE` IOBs are free. With 
`CONFIG_NET_TCP_WRITE_BUFFERS`, the write buffers can use all IOBs above the 
throttle, and the received segment holds some of the others. When the peer is 
on this host (127.0.0.1, or a local address of a network device), the write 
buffers that hold the IOBs are the buffers of that peer, and they free their 
IOBs only when the dropped data is acknowledged. Every segment is dropped and 
the transfer stops. With the default pool (96 IOBs of 196 bytes), a TCP 
transfer over 127.0.0.1 stops after about 20 KB, also between two threads of 
one process; on master the sender resets the connection after about 13.5 s.
   
   A log at each drop shows the pool at that time: 1 IOB free, 87 IOBs in the 
write queue of the sender, 8 IOBs in the received segment, the read-ahead 
buffer of the receiver empty, the drop in `tcp_data_event()`.
   
   The throttle keeps IOBs for sending and for the RX buffers of the drivers. A 
segment of a peer on this host comes through `devif_loopback()`, not from the 
RX buffer of a driver. This PR keeps the data of such a peer as before 
ea5d2c9ae49 (new `tcp_peer_is_local()`: the segment came through the loopback 
device, or the remote address of the connection is an address of the device). 
The data of other peers is dropped below the throttle as before, so the case of 
#22000 does not change.
   
   Another possible fix is the rule that `tcp_datahandler()` had until 
34d2cde8a8f (an empty read-ahead buffer takes the data, bf210560017). It was 
tested and is not sufficient: the transfer goes on at about 5 KB/s, because the 
segments that arrive while the read-ahead buffer holds one segment are dropped 
and sent again after the retransmission timeout.
   
   ## Impact
   
   TCP with `CONFIG_IOB_THROTTLE > 0` (the default with write buffers and 
read-ahead): the received data of a connection whose peer is on this host is no 
longer dropped below the throttle. Other connections, UDP and `IOB_THROTTLE` 0 
do not change. No API, ABI or Kconfig change.
   
   For such a connection, the read-ahead buffer can again take IOBs below the 
throttle, as before ea5d2c9ae49. The sender on this host takes the IOBs of each 
segment from the free pool, so it cannot send more when the pool is empty; the 
IOBs come back when the application reads.
   
   Build modes and architectures: common network code, the same in flat, 
protected and kernel builds; SMP: the check runs under the same locks as the 
drop before.
   
   ## Testing
   
   qemu-armv8a under HVF (Apple M-series), TCP over the loopback device, 96 
IOBs of 196 bytes, `IOB_THROTTLE` 8, `CONFIG_NET_TCP_WRITE_BUFFERS` unless 
noted; master ed122374690 (the PR is rebased on 11c71e16384 after the runs, 
with no change in `net/`; the final head, which also takes every segment of the 
loopback device as local, was run again with `cmocka_net_tcp`, the reproducers, 
`tcpthru` and ostest on knsh, 1 CPU, with the same results), the parent of 
#22000 is 9df78aa1e69. The programs run from `/system/bin` (kernel build).
   
   - `tcplomin`: one thread sends 8 MiB to another thread of the same process 
over 127.0.0.1 (the program of #22088).
   - `tcplobulk`: the same with a progress line every second and a 60 s limit; 
also with 4 KiB `send()`/`recv()`.
   - `tcpthru loop 20`: 20 MiB between two threads.
   - `cmocka_net_tcp` with the new case `test_tcp_loopback_bulk` 
(apache/nuttx-apps#APPS): 4 MiB from a thread with blocking `send()`, 30 s 
limit (an earlier 1 MiB version on the parent and on ea5d2c9ae49).
   
   | Test (knsh, HVF) | 9df78aa1e69 (before #22000) | ea5d2c9ae49 | master | 
this PR |
   |---|---|---|---|---|
   | `tcplomin`, 1 CPU | PASS | not done after 300 s | FAIL: `send()` gives 
`ENOTCONN`, 14674 of 8 MiB received | PASS |
   | `tcplobulk`, 64 KiB calls, 1 CPU | PASS, 79.9 MiB/s | about 4.5 KB/s | 
FAIL at 14674 bytes (`ECONNRESET`) | PASS, 79.6 MiB/s |
   | `tcplobulk`, 4 KiB calls, 1 CPU | PASS, 79.8 MiB/s | | FAIL at 18540 bytes 
| PASS, 79.3 MiB/s |
   | `tcpthru loop 20`, 1 CPU | PASS, 46.0 MiB/s | | FAIL: `ENOTCONN` after 
13.5 s | PASS, 46.5 MiB/s |
   | `test_tcp_loopback_bulk`, 1 CPU, 2 runs | PASS (1 MiB version) | FAIL: 
63008 of 1 MiB after 31 s (1 MiB version) | FAIL: `ECONNRESET` after 23330 
bytes, 13 s | PASS |
   | `tcplobulk` to the address of eth0 (virtio-net), 1 CPU | | | FAIL at 17308 
bytes (`ECONNRESET`) | PASS, 79.3 MiB/s |
   | `tcplomin`, 4 CPUs | | | FAIL: 223106 bytes received | PASS |
   | `tcpthru loop 20`, 4 CPUs | | | FAIL: `ENOTCONN` after 13.5 s | PASS, 
455.9 MiB/s |
   | `test_tcp_loopback_bulk`, 4 CPUs | | | FAIL in 4 of 4 runs (2 boots): 
`ECONNRESET` after 70 KB to 2 MB | PASS in 2 of 2 runs |
   | flat qemu-armv8a:nsh: `test_tcp_loopback_bulk` | | | FAIL: `ECONNRESET` 
after 23330 bytes | PASS |
   | no TCP write buffers (`IOB_THROTTLE` 8): all rows of 1 CPU | | | PASS | 
PASS |
   | `IOB_THROTTLE` 0: `tcplomin`, `tcplobulk` | | | | PASS |
   | `ostest` (each boot) | status 0 | status 0 | status 0 | status 0 |
   
   A slow reader (1 KiB `recv()` and a 2 ms pause) gets about 4 to 16 KB/s over 
127.0.0.1 with write buffers, before #22000 and with this PR alike (master 
resets the connection); this is not changed here. `test_tcp_connect_ipv4` of 
`cmocka_net_tcp` fails in this configuration on all versions (`ECONNREFUSED` 
from the group server).
   
   #22000's test (4 host connections echo 256 KiB each through a NuttX node at 
the same time, random writes of 1 to 4096 bytes; sim without write buffers, 100 
IOBs of 196 bytes, `IOB_THROTTLE` 8, `examples/tcpecho`):
   
   | | master | this PR | master without ea5d2c9ae49 |
   |---|---|---|---|
   | connections that echoed all 256 KiB (6 runs x 4) | 5 of 24 | 4 of 24 | 0 
of 24 |
   | free IOBs after the runs | 100 of 100 | 100 of 100 | 1 of 100 |
   
   Without ea5d2c9ae49 the received data keeps the whole pool and every 
connection stops; with it, on master and with this PR, the pool comes back. 
With `examples/tcpecho` (one thread, 64-byte reads) no run completes all 4 
connections, also on master (the other connections stop, and later ones find no 
free TCP connection), so these runs show the pool, not the pass rate of #22000. 
The peers of this test are not on this host, so the changed condition does not 
apply to them.
   
   Builds in the NuttX CI image (`EXTRAFLAGS="-Wno-cpp -Werror"`), master and 
this PR: all OK, 0 warnings. Text size (bytes):
   
   | Configuration | Build mode | `IOB_THROTTLE` | master | this PR |
   |---|---|---|---|---|
   | qemu-armv8a:netnsh (no TCP write buffers) | flat | 8 | 554132 | 554148 
(+16) |
   | qemu-armv8a:netnsh + `NET_TCP_WRITE_BUFFERS` | flat | 8 | 557108 | 557124 
(+16) |
   | the same, CMake | flat | 8 | 557908 | 557924 (+16) |
   | the same + `CONFIG_IOB_THROTTLE=0` | flat | 0 | 556856 | 556856 (+0) |
   | qemu-armv8a:netnsh + write buffers + `NET_IPv6` (both domains) | flat | 8 
| 569780 | 569832 (+52) |
   | qemu-armv8a:netnsh_smp + write buffers | flat, SMP | 8 | 532804 | 532820 
(+16) |
   | rv-virt:pnsh64 + TCP, write buffers | protected | 8 | 184672 | 184672 (+0) 
|
   | rv-virt:knetnsh64 | kernel | 2 | 229103 | 229109 (+6) |
   | rv-virt:knetnsh64 + write buffers, dynamic pools | kernel | 2 | 231483 | 
231489 (+6) |
   | lc823450-xgevk:krndis | protected | 0 | 193476 | 193476 (+0) |
   | sim:tcpblaster (IPv4 and IPv6) | sim | 16 | 997922 | 998066 (+144) |
   | sim:tcpblaster without write buffers | sim | 16 | 989234 | 989378 (+144) |
   | lm3s6432-s2e:nsh | flat | 0 | 64932 | 64932 (+0) |
   
   IPv6 only is not built here: qemu-armv8a:netnsh selects the IPv4 DHCP 
client, which does not build without IPv4 (master too). The IPv6 condition is 
built in the two dual-stack rows.
   
   `tools/checkpatch.sh -c -u -m -g` passes. Documentation: no page describes 
this drop (`Documentation/os/memory/iob.rst` describes `IOB_THROTTLE` in 
general terms that stay true).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to