Hello,
I'm looking for help understanding the way libcurl reuses connections and the
reasons for it closing connections which should be reusable for sending HTTP
requests
(HTTP 1.1 protocol).
I'm using libcurl in a program for sending many HTTP requests in parallel. I
use libcurl with libev, loosely based on the example in
https://curl.haxx.se/libcurl/c/evhiperfifo.html.
I use libcurl version 7.65.3.
I use libcurl to send HTTP requests at a constant rate for some minutes
(between 8 - 30 minutes). The destination is always an explicit IP address (no
DNS resolution needed).
My problem is that during my test runs, libcurl closes connections, and later
opens new connections to the same destinations.
I set option CURLMOPT_MAXCONNECTS with a very large number, and so connections
don't close because of a limitation on the max size of the connection's cache.
In a setup with Windows OS, I send ~60 HTTP requests per second, all to the
same constant IP address. Even though the load stays constant for minutes,
libcurl occasionally
closes a connection because it hasn't been used for too long (in function
"conn_maxage"), and shortly afterwards it opens another connection to the same
destination.
I know libcurl has multiple connections open towards the same destination due
to the load. How come it doesn't use one of them for almost 2 minutes, leading
to its closure? libcurl did send many more requests during the 2 minutes than
the number of open connections. I assumed that libcurl chooses which connection
to reuse
in a round-robin manner. Is it not so? How libcurl chooses which of the open
connections to use when it sends a request? Is this behavior influenced by some
CURL option? Can someone please direct me to the code where libcurl chooses a
free connection from the cache for sending a request?
In another setup with Linux OS, I need to send hundreds (e.g. 600) of HTTP
requests per second. There are 10 destination IP addresses. I send requests to
them
in a roughly balanced manner. libcurl also closes connections in this case;
this time, it does it both from function "conn_maxage", and from function
"multi_done"
(("conn->bits.close" is true, although I didn't use an option to prevent
libcurl from reusing connections, and
"(premature && !(conn->handler->flags & PROTOPT_STREAM))" is true). I read the
comments describing these conditions, but I don't really understand...
Can someone please explain why libcurl closes the connection in this case?
Thanks,
Dorit
This email and any files transmitted with it are confidential material. They
are intended solely for the use of the designated individual or entity to whom
they are addressed. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, use, distribution or
copying of this communication is strictly prohibited and may be unlawful.
If you have received this email in error please immediately notify the sender
and delete or destroy any copy of this message
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
