On 27-Jan-22 07:47, James Read via curl-library wrote:
On Thu, Jan 27, 2022 at 12:06 PM Henrik Holst <henrik.ho...@millistream.com> wrote:I wonder if the results that you see from that example is due to the short life of each connection, aka most of the time there is spent on the tcp-handshake (and possible the tls handshake on top of it all) combined with a small initial tcp window that never gets large enough before the page has been downloaded in full. I have personally no experience pushing curl to its bw limits (I have no such use case) but I do use epoll for raw tcp/ip solutions where I easily push Gbps per connection but that is for very long lived connections (days and months) so a whole different ballpark. Try to download a much larger file than just the front page and see if that changes things.I have experimented with wrk. I have modified the codebase to connect to multiple servers rather than a single server. I am able to saturate the bandwidth as long as connections are reused. As soon as I force to program to make single use of a connection the throughput falls off a cliff edge. I am beginning to think there is a problem with the TCP/IP stack implementation that limits usage with many connections.James Read
Depending on the server, and any middlebox issues, TFO (TCP Fast Open) MAY help. There's a CURLOPT to enable it.
Note that when it doesn't help, it hurts. You should read up on it and not blindly enable. And probably keep track of the servers that support it. When it works, data is sent with the SYN. When it doesn't, in the worst case, the connection times out and the attempt starts over.
Timothe Litt ACM Distinguished Engineer -------------------------- This communication may not represent the ACM or my employer's views, if any, on the matters discussed.
OpenPGP_signature
Description: OpenPGP digital signature
-- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
