RE: Network starvation question
Cheng is correct; A non-reactive UDP flow (an application pushing data as quickly as it can, without any regards if the packet even departs the machine) will always be able to ursurp excessive amounts of network capacity. TCP uses (well-designed) congestion control, in order to prevent TCP from ursurping excessive amounts of resources - but that also means in the presence of a bully, it will yield excessively. If you can not control that aggressive application directly, the canonical approach is to introduce some form of AQM (active queue manangment) on your bottleneck queue; E.g. FQ-Codel - with the FQ part making sure, that there is an effective minimum bandwidth guarantee for very light flows at all times (DNS, NTP, RDP, SSH) even in the presence of unreactive, aggressive and bandwidth-hogging flows (regardless of protocol; meaning this approach works not only with UDP, or a misused CC in TCP, but also SCTP, QUIC, RTMP and all others). Best regards, Richard From: owner-freebsd-...@freebsd.org On Behalf Of Cheng Cui Sent: Freitag, 3. November 2023 12:53 To: Yuri Cc: freebsd-net@freebsd.org Subject: Re: Network starvation question Hi Yuri, If I understand your situation correctly, your application A is using UDP but application B is using TCP and the max outbound bandwidth is < 4MBps. If A can send at 3.5 Mbps and B can send at 0.5 Mbps, why B is suffering more in throughput when the outbound limit is < 4Mbps? There are many factors I think in my experience that can contribute to this. But maybe we prefer a solution rather than the root cause of the difference. If you can tune A to use less bandwidth and let B to use the full 0.5 Mbps, the problem may be solved. (for example, iperf default in UDP is ~1Mbps but can be tuned to send at max bandwidth, so vice versa) Then, back to the possible contribution factors of TCP suffering from a UDP traffic competition : 1. UDP traffic rate can be considered constant so it does not yield 2. TCP congestion control may be encountered 3. application's responsiveness may be different Best Regards, Cheng Cui On Fri, Nov 3, 2023 at 12:42 AM Yuri wrote: Hi, I've encountered the situation when the application A was using 100% of the outbound bandwidth which is approximately 3.5 MBps of UDP traffic. Then the application B (the RDP TCP connection) attempted to use a much lower outbound speed, probably < 0.5 MBps, and it got starved. Application B (RDP) was super slow as long as the application A kept running. It was almost impossible to use the RDP connection. My question is: shouldn't the system allow less intense streams to also run at a decent speed? Let's say that the outbound bandwidth threshold of the connection is 3.5 MBps. The application A can send 3.5 MBps (or more). The application B can send up to 0.5 MBps. Obviously, they can't send 4.0 MBps in total, and their speeds should be tuned down. If both of the applications would be tuned down proportionately, this could be done using the 3.5/4.0 ratio, which would be 0.875. So why then does the slower connection get slowed down so much? It was obviously slowed down many times, not just by 13%. FreeBSD 13.2 Thanks, Yuri OpenPGP_0x17BE5899E0B1439B.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Network starvation question
On 11/4/23 03:37, Yuri wrote: Router is also involved, and the provider's network as well. But I think that there is a bug in the FreeBSD's network code that it allows a slower TCP connection to be hammered like this, unless there is a good explanation for this observation. Yuri As you mention router and provider network are involved my first thought is bufferbloat. See https://en.wikipedia.org/wiki/Bufferbloat As UDP is involved FreeBSD doesn't do any flow control on that traffic and just sends as much as the application requests it to do. This is not a bug in FreeBSD. The design of the application is to requests FreeBSD to do this by using UDP instead of TCP. A solution to this would be to limit bandwidth of application A (mentioned in your original mail) to leave just enough space for application B to perform well. Maybe the application has support for this. Otherwise a mechanism like "dummynet" (in combination with ipfw firewall) can do this. Regards, Ronald.
Re: Network starvation question
Re: Network starvation question
Router is also involved, and the provider's network as well. But I think that there is a bug in the FreeBSD's network code that it allows a slower TCP connection to be hammered like this, unless there is a good explanation for this observation. Yuri
Re: Network starvation question
Hi Yuri, If I understand your situation correctly, your application A is using UDP but application B is using TCP and the max outbound bandwidth is < 4MBps. If A can send at 3.5 Mbps and B can send at 0.5 Mbps, why B is suffering more in throughput when the outbound limit is < 4Mbps? There are many factors I think in my experience that can contribute to this. But maybe we prefer a solution rather than the root cause of the difference. If you can tune A to use less bandwidth and let B to use the full 0.5 Mbps, the problem may be solved. (for example, iperf default in UDP is ~1Mbps but can be tuned to send at max bandwidth, so vice versa) Then, back to the possible contribution factors of TCP suffering from a UDP traffic competition : 1. UDP traffic rate can be considered constant so it does not yield 2. TCP congestion control may be encountered 3. application's responsiveness may be different Best Regards, Cheng Cui On Fri, Nov 3, 2023 at 12:42 AM Yuri wrote: > Hi, > > > I've encountered the situation when the application A was using 100% of > the outbound bandwidth which is approximately 3.5 MBps of UDP traffic. > > Then the application B (the RDP TCP connection) attempted to use a much > lower outbound speed, probably < 0.5 MBps, and it got starved. > > Application B (RDP) was super slow as long as the application A kept > running. It was almost impossible to use the RDP connection. > > > My question is: shouldn't the system allow less intense streams to also > run at a decent speed? > > > Let's say that the outbound bandwidth threshold of the connection is 3.5 > MBps. > > The application A can send 3.5 MBps (or more). > > The application B can send up to 0.5 MBps. > > Obviously, they can't send 4.0 MBps in total, and their speeds should be > tuned down. > > If both of the applications would be tuned down proportionately, this > could be done using the 3.5/4.0 ratio, which would be 0.875. > > So why then does the slower connection get slowed down so much? > > It was obviously slowed down many times, not just by 13%. > > > FreeBSD 13.2 > > > Thanks, > > Yuri > > > >
Network starvation question
Hi, I've encountered the situation when the application A was using 100% of the outbound bandwidth which is approximately 3.5 MBps of UDP traffic. Then the application B (the RDP TCP connection) attempted to use a much lower outbound speed, probably < 0.5 MBps, and it got starved. Application B (RDP) was super slow as long as the application A kept running. It was almost impossible to use the RDP connection. My question is: shouldn't the system allow less intense streams to also run at a decent speed? Let's say that the outbound bandwidth threshold of the connection is 3.5 MBps. The application A can send 3.5 MBps (or more). The application B can send up to 0.5 MBps. Obviously, they can't send 4.0 MBps in total, and their speeds should be tuned down. If both of the applications would be tuned down proportionately, this could be done using the 3.5/4.0 ratio, which would be 0.875. So why then does the slower connection get slowed down so much? It was obviously slowed down many times, not just by 13%. FreeBSD 13.2 Thanks, Yuri
