I have a simple C program where I use the CURL multi interface to download a bunch of files. My code looks like this:
#include <stdlib.h> #include <stdio.h> #include <curl/curl.h> int main(void) { int code = 0; size_t index = 0; int running = 1; CURLMsg* msg = NULL; int left = 0; CURL* curl = NULL; CURLM* curl_multi = NULL; curl = curl_easy_init(); curl_multi = curl_multi_init(); curl_easy_setopt(curl, CURLOPT_URL, "https://example.com/"); curl_easy_setopt(curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); curl_multi_setopt(curl_multi, CURLMOPT_MAX_HOST_CONNECTIONS, 5L); for (index = 0; index < 15; index++) { CURL* handle = curl_easy_duphandle(curl); curl_multi_add_handle(curl_multi, handle); } while (running) { curl_multi_perform(curl_multi, &running); if (running) { curl_multi_poll(curl_multi, NULL, 0, 1000, NULL); } while ((msg = curl_multi_info_read(curl_multi, &left))) { if (msg->msg != CURLMSG_DONE) { continue; } curl_multi_remove_handle(curl_multi, msg->easy_handle); printf("result: %i\n", (int) msg->data.result); } } return 0; } I firstly create an easy handle and then duplicate that handle with curl_easy_duphandle() for usage with the multi interface. For each handle, I set the CURLOPT_URL to the URL of the file I want to download. I set a maximum number of host connections here (CURLMOPT_MAX_HOST_CONNECTIONS) because I don't want to cause overhead on the server. This code does what I intended: download up to 5 files simultaneously. However, things get a little weird if I set a CURLOPT_DOH_URL on the main easy handle: ... curl_easy_setopt(curl, CURLOPT_DOH_URL, "https://dns.google/dns-query"); ... This causes the multi interface to hang indefinitely. I tried setting CURLOPT_VERBOSE to discover what's going on, and here are the logs: * Found bundle for host: 0xb4000073436429d0 [serially] * Found bundle for host: 0xb4000073436429d0 [serially] * Found bundle for host: 0xb4000073436429d0 [serially] * Found bundle for host: 0xb4000073436429d0 [serially] * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. * Found bundle for host: 0xb4000073436429d0 [serially] * No more connections allowed to host: 5 * No connections available. It then stays like this. No requests are performed. After some tests, I found that setting CURLMOPT_MAX_HOST_CONNECTIONS to anything higher than 10 resolves this issue: ``` curl_multi_setopt(curl_multi, CURLMOPT_MAX_HOST_CONNECTIONS, 10L); ``` However, this setting causes the multi interface to make 10 requests simultaneously instead of just 5, which is undesired for my specific use case. Here is the output of curl_version(): libcurl/8.7.1 OpenSSL/1.0.2k-fips zlib/1.2.7 brotli/1.0.9 libidn2/2.3.7 libpsl/0.7.0 (+libicu/50.1.2) libssh2/1.10.0 nghttp2/1.33.0 OpenLDAP/2.4.44 I'm on CentOS 7.
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html