Re: configure options for various libcurl builds

2021-08-21 Thread Daniel Stenberg via curl-library
On Sat, 21 Aug 2021, Anmol Mishra via curl-library wrote: Could there be some guidance about a simple libcurl static build for For configure builds, both a shared and a static libcurl version are built. SMTP is enabled by default. -- / daniel.haxx.se | Commercial curl support up to 24x7

Re: configure options for various libcurl builds

2021-08-21 Thread Dan Fandrich via curl-library
On Sat, Aug 21, 2021 at 05:42:47PM +, Anmol Mishra via curl-library wrote: > Could there be some guidance about a simple libcurl static build for Static building is tricky because the entire chain of static dependencies resulting from all libcurl dependencies needs to be added to the fi

configure options for various libcurl builds

2021-08-21 Thread Anmol Mishra via curl-library
Could there be some guidance about a simple libcurl static build for a. send email via smtp and b. downloading a file from a url These may be the majority of uses for libcurl. I tried the various ./configure options for a couple of days - but its not trivial to isolate just these two functions

Re: libcurl hangs in curl_multi_cleanup

2021-06-23 Thread Daniel Stenberg via curl-library
On Wed, 23 Jun 2021, firefo...@interia.pl wrote: So, since 215db086e0 test660 ensures wrong behavior of libcurl. Since we try to have tests for *everything* curl can do, so the fact that we verify bugs is what is *supposed* to happen. Until we realize it is a bug, it isn't a bug

Re: libcurl hangs in curl_multi_cleanup

2021-06-23 Thread FirefoxOS--- via curl-library
ect" So, since 215db086e0 test660 ensures wrong behavior of libcurl. -- --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html

Re: libcurl hangs in curl_multi_cleanup

2021-06-23 Thread Daniel Stenberg via curl-library
On Tue, 22 Jun 2021, FirefoxOS via curl-library wrote: Thanks for creating a test case. I've now made a PR that I believe fixes this issue: https://github.com/curl/curl/pull/7288 ... and the test 677 is included there and reproduces the bug before this fix. -- / daniel.haxx.se |

Re: libcurl hangs in curl_multi_cleanup

2021-06-22 Thread FirefoxOS via curl-library
doesn't seem to accept commands terminated with just \n, contrary to the server I used before.) What I wasn't aware of before I saw your test677 file is that after libcurl "finishes" CONNECT_ONLY connection and passes ACTIVE_SOCKET to the application, it still wants to proceed

Re: libcurl hangs in curl_multi_cleanup

2021-06-18 Thread Daniel Stenberg via curl-library
On Fri, 18 Jun 2021, Firefox OS via curl-library wrote: Do you mean imap://...? Yes, it does. And the callstack looks the same. I cannot reproduce. I converted "curl_imap_teardown_issue.c" into test case 677 in this commit:

Re: libcurl hangs in curl_multi_cleanup

2021-06-18 Thread Firefox OS via curl-library
Dnia 2021-06-18, o godz. 09:31:42 Daniel Stenberg napisał(a): > On Thu, 17 Jun 2021, FirefoxOS via curl-library wrote: > > > Can you help me understand what does test660 actually check? > > > The issue is easily reproducible using curl_imap_teardown_issue.c > > Then clearly test660

Re: libcurl hangs in curl_multi_cleanup

2021-06-18 Thread Daniel Stenberg via curl-library
On Thu, 17 Jun 2021, FirefoxOS via curl-library wrote: Can you help me understand what does test660 actually check? The issue is easily reproducible using curl_imap_teardown_issue.c Then clearly test660 doesn't run the same sequence. Does the curl_imap_teardown_issue.c code also cause a

Re: libcurl hangs in curl_multi_cleanup

2021-06-17 Thread FirefoxOS via curl-library
hat test case > still runs fine today. > Hello, Can you help me understand what does test660 actually check? Do I understand correctly that it executes libtest/lib597.c? Does it check if the cleanup sequence documented at https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html hangs insi

webinar: common libcurl mistakes

2021-06-15 Thread Daniel Stenberg via curl-library
Hello friends! This is just a quick mention that I'll do a free-to-attend webinar this Tursday titled as the subject says: common libcurl mistakes. With a following Q of course. Sign up here: https://www.wolfssl.com/upcoming-wolfssl-webinar-common-libcurl-mistakes/ See you Thursday

Re: libcurl hangs in curl_multi_cleanup

2021-06-10 Thread Daniel Stenberg via curl-library
On Thu, 10 Jun 2021, Firefox OS wrote: The story is the same as 2 years ago You perceive it as the same, I do not. The previous issue was reproduced with and then verified with test 660 and that test case still runs fine today. -- / daniel.haxx.se | Commercial curl support up to 24x7 is

Re: libcurl hangs in curl_multi_cleanup

2021-06-10 Thread Firefox OS via curl-library
Dnia 2021-06-10, o godz. 08:14:05 Daniel Stenberg napisał(a): > On Thu, 10 Jun 2021, Firefox OS via curl-library wrote: > > > Unfortunately the bug is there again. > > At least in 7.76.1 and 7.70.0. > > Particularly, curl_multi_cleanup() hangs for exactly 2 minutes. > > Please tell us the

Re: libcurl hangs in curl_multi_cleanup

2021-06-10 Thread Daniel Stenberg via curl-library
On Thu, 10 Jun 2021, Firefox OS via curl-library wrote: Unfortunately the bug is there again. At least in 7.76.1 and 7.70.0. Particularly, curl_multi_cleanup() hangs for exactly 2 minutes. Please tell us the whole story. What do you do and when does it hang? We want to be able to reproduce.

Re: libcurl hangs in curl_multi_cleanup

2021-06-09 Thread Firefox OS via curl-library
fix for this issue is now here: > > > > > > https://github.com/curl/curl/pull/3795 > > > > That one landed in git too and will ship in the next release. > > > > Hi, > > I've rebuilt libcurl-7.64.1 patched with just > https://github.com/curl/curl

Re: libcurl options for --create-dirs -O --output-dir

2021-06-08 Thread Daniel Stenberg via curl-library
On Tue, 8 Jun 2021, Anmol Mishra via curl-library wrote: However, I cannot see the equivalent options inside libcurl. Is the --output-dir option only available in the curl CLI ? Yes, since libcurl doesn't really save content to files/directories. It delivers the content to a callback

libcurl options for --create-dirs -O --output-dir

2021-06-08 Thread Anmol Mishra via curl-library
Daniel posted about the --output-dir option in curl 7.73 - https://daniel.haxx.se/blog/2020/09/10/store-the-curl-output-over-there/ However, I cannot see the equivalent options inside libcurl. Is the --output-dir option only available in the curl CLI ? [https://daniel.haxx.se/blog/wp-content

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-06-07 Thread Daniel Stenberg via curl-library
On Mon, 7 Jun 2021, Jeroen Ooms wrote: I'm curious to know if this was followed up on? R users have been reporting random HTTP2 framing layer errors for multiplexed connections to google servers/apis for a long time, but I have never succeeded to narrow down the exact problem.

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-06-07 Thread Jeroen Ooms via curl-library
On Thu, Jun 3, 2021 at 7:56 PM Daniel Stenberg via curl-library wrote: > > On Thu, 3 Jun 2021, Shikha Sharma wrote: > > > Proposed solution: > > Ah, yes that looks interesting - especially since it fixes the problem for > you! Are you able to make a pull-request out of that and submit, or would

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-06-03 Thread Daniel Stenberg via curl-library
On Thu, 3 Jun 2021, Shikha Sharma wrote: Proposed solution: Ah, yes that looks interesting - especially since it fixes the problem for you! Are you able to make a pull-request out of that and submit, or would you like me to do it for you? Thanks! -- / daniel.haxx.se | Commercial curl

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-06-03 Thread Shikha Sharma via curl-library
Proposed solution: curl -V curl 7.74.0 (x86_64-redhat-linux-gnu) libcurl/7.74.0 NSS/3.44 zlib/1.2.7 libpsl/0.7.0 (+libicu/50.1.2) libssh2/1.8.0 nghttp2/1.39.1 Release-Date: 2020-12-09 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-05-27 Thread Daniel Stenberg via curl-library
On Thu, 27 May 2021, Shikha Sharma via curl-library wrote: HTTP2 connection with prior knowledge initiated by curl, No TLS. Server sends GoAway frame after processing each request. I suspect the problem/bug here is that the request that reads the trailing GOAWAY isn't properly considered as

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-05-27 Thread Shikha Sharma via curl-library
Thanks for your reply. I tried it with the latest version and unfortunately it still doesnt work. curl -V curl 7.77.0 (x86_64-redhat-linux-gnu) libcurl/7.77.0 NSS/3.53.1 zlib/1.2.7 libpsl/0.7.0 (+libicu/50.1.2) libssh2/1.9.0 nghttp2/1.39.1 OpenLDAP/2.4.44 libmetalink/0.1.3 Release-Date

Re: Libcurl reports 'Error in the HTTP2 framing layer' (16) for outgoing request.

2021-05-27 Thread Daniel Stenberg via curl-library
On Thu, 27 May 2021, Shikha Sharma via curl-library wrote: I am facing an issue where curls responds with error "'Error in the HTTP2 framing layer' (16)" to the application. Curl version: libcurl-7.74.0 Scenario: Sending multiple outgoing HTTP2 requests to an external HTTP ser

Re: libcurl with custom malloc and SSL

2021-05-27 Thread Daniel Stenberg via curl-library
On Thu, 27 May 2021, Ville Likitalo wrote: it was a build/linking related one instead. Great, glad you found it and that it wasn't a curl bug! =) -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features |

Re: libcurl with custom malloc and SSL

2021-05-27 Thread Ville Likitalo via curl-library
> >allocated with the private malloc libcurl has been set up with. > > > >Now, I could not quickly find any existing open or closed bug > >that would describe this but would this already be fixed in the > >later versions? > > Without debug symbol

Re: libcurl with custom malloc and SSL

2021-05-25 Thread Daniel Stenberg via curl-library
On Tue, 25 May 2021, Ville Likitalo via curl-library wrote: The only interesting bit in the stack trace is that the pointer which gets passed to libc free(void*) is one that has been allocated with the private malloc libcurl has been set up with. Now, I could

libcurl with custom malloc and SSL

2021-05-25 Thread Ville Likitalo via curl-library
Hi, I am trying out libcurl 7.64.0 (right now stuck with that version for political reasons) with custom malloc implementation and SSL. This was the result: [Thread 0x7fffd3fff700 (LWP 147) exited] *** Error in `/...': free(): invalid pointer

Re: Crash libCurl 7.76.1 32-bit build in H3 transfers

2021-05-06 Thread Daniel Stenberg via curl-library
On Thu, 6 May 2021, Dmitry Karpov via curl-library wrote: So "size_t datalen" in libCurl is different from "uint64_t datalen" in the ngtcp2 library. Fix pending: https://github.com/curl/curl/pull/7027 The crash was so obvious, so it was a bit surprising how it was able to

Crash libCurl 7.76.1 32-bit build in H3 transfers

2021-05-06 Thread Dmitry Karpov via curl-library
Hello, I stepped on a crash in libCurl 7.76.1 32-bit in H3 transfers (MSVC, Win32). It occurred on any direct H3 transfers from public H3 servers referred on Curl web page (https://bagder.github.io/HTTP3-test/) I debugged the issue and it seems to be related to 32-bit builds and not Windows

Re: POC for ECH support in curl and libcurl

2021-05-02 Thread Daniel Stenberg via curl-library
On Sun, 2 May 2021, Niall O'Reilly via curl-library wrote: I would like to let people know of a proof-of-concept implementation of ECH in curl and libcurl. This uses OpenSSL as back-end and interoperates with Cloudflare’s demonstration server. For more information, please see https

POC for ECH support in curl and libcurl

2021-05-02 Thread Niall O'Reilly via curl-library
Hello. I would like to let people know of a proof-of-concept implementation of ECH in curl and libcurl. This uses OpenSSL as back-end and interoperates with Cloudflare’s demonstration server. For more information, please see https://github.com/niallor/curl/blob/ECH-WIP/docs/ECH.md Best regards

Re: libcurl LDAP in asyncmod

2021-05-02 Thread Fulup Ar Foll via curl-library
Patrick, It is not as clear as that, if you have a wrong LDAP request schema, ldap returned code is also store into msg->result, when in my opinion this is an LDAP protocol error. In any case it allows to retrieve an classify low level errors as: invalid-credential, invalid-request, ...

Re: libcurl LDAP in asyncmod

2021-05-01 Thread Patrick Monnerat via curl-library
On 5/1/21 10:15 PM, Fulup Ar Foll wrote: @patrick:  retrieving connection code with LDAP works in both synchronous and asynchronous mode and not only in HTTP,TFTP,... There are two things: - the curl result code of type CURLcode: this one is always available as this is the value

Re: libcurl LDAP in asyncmod

2021-05-01 Thread Daniel Stenberg via curl-library
On Sat, 1 May 2021, Fulup Ar Foll wrote: As the error status exist in synchronous mode, it is obvious that you handle it somewhere. I would have hope to get it from curl_easy_getinfo with something like:     curl_easy_getinfo(httpRqt->easy, CURLINFO_x_x, ); Unfortunately from what I

Re: libcurl LDAP in asyncmod

2021-05-01 Thread Fulup Ar Foll via curl-library
Daniel, As the error status exist in synchronous mode, it is obvious that you handle it somewhere. I would have hope to get it from curl_easy_getinfo with something like:     curl_easy_getinfo(httpRqt->easy, CURLINFO_x_x, ); Unfortunately from what I understand the code

Re: libcurl LDAP in asyncmod

2021-04-30 Thread Daniel Stenberg via curl-library
On Fri, 30 Apr 2021, Fulup Ar Foll via curl-library wrote: + curl_multi_info_read return CURLMSG_DONE Allow me to quote curl_multi_info_read.3: When msg is CURLMSG_DONE, the message identifies a transfer that is done, and then result contains the return code for the easy handle

Re: libcurl LDAP in asyncmod

2021-04-30 Thread Fulup Ar Foll via curl-library
ODE, >status); return ZERO Question: how to retrieve conection-deny status in asynchronous mode ? From the documentation at https://curl.se/libcurl/c/CURLINFO_RESPONSE_CODE.html, CURLINFO_RESPONSE_CODE is only available for http, ftp and smtp, not for ldap. In other words, you

Re: libcurl LDAP in asyncmod

2021-04-30 Thread Patrick Monnerat via curl-library
On 4/30/21 4:58 PM, Fulup Ar Foll via curl-library wrote: curl_easy_getinfo(httpRqt->easy, CURLINFO_RESPONSE_CODE, >status); return ZERO Question: how to retrieve conection-deny status in asynchronous mode ? From the documentation at https://curl.se/libcurl/c/CURLINFO_RESPONSE_COD

libcurl LDAP in asyncmod

2021-04-30 Thread Fulup Ar Foll via curl-library
I try to use libcurl for retrieving group list from a user from C/API.  It works both in synchronous and asynchronous, nevertheless in asynchronous mode, status is not position when credential are wrong  in asynchrounous: curl_easy_perform return

Re: Cannot build program with linking libcurl as static lib.

2021-04-10 Thread Daniel Stenberg via curl-library
On Sat, 10 Apr 2021, Mateusz Domińczak via curl-library wrote: curl-7.76.0 libcurl-vc16-x86-release-static-ipv6-sspi-schannel (so static as we see) Visual Studio 19. Using the winbuild makefile I presume? Can you tell us your full command line you used for the build? It might make it easier

Cannot build program with linking libcurl as static lib.

2021-04-10 Thread Mateusz Domińczak via curl-library
My build is: curl-7.76.0 libcurl-vc16-x86-release-static-ipv6-sspi-schannel (so static as we see) Visual Studio 19. Linked all required libraries: Normaliz.lib;Ws2_32.lib;Wldap32.lib;Crypt32.lib;advapi32.lib; (And libcurl ofc) Defined using #define CURL_STATICLIB (defined before the include

Re: Getting pubkey fingerprint in libcurl

2021-03-14 Thread Ray Satiro via curl-library
On 3/12/2021 7:13 AM, Morten Minde Neergaard wrote: At 01:02, Tue 2021-03-02, Ray Satiro via curl-library wrote: On 2/26/2021 4:10 PM, Morten Minde Neergaard via curl-library wrote: [...] It would be a lot more elegant if there were an option to get the pubkey fingerprint directly, using the

A libcurl internals presentation should contain...

2021-03-12 Thread Daniel Stenberg via curl-library
Hi friends! I'm working on slides and material for a "libcurl internals" presentation. The intended audience is developers who are interested in learning how the libcurl engine works and some of the internal concepts and designs, to perhaps help understand on how to work with it w

Re: Getting pubkey fingerprint in libcurl

2021-03-01 Thread Ray Satiro via curl-library
On 2/26/2021 4:10 PM, Morten Minde Neergaard via curl-library wrote: I'm making an app that's using public key pinning, and it would be very helpful to have programmatic access to the pubkey fingerprint. The app currently has a huge and horrible mountain of platform- and backend-specific code

Getting pubkey fingerprint in libcurl

2021-02-26 Thread Morten Minde Neergaard via curl-library
Hi, I'm making an app that's using public key pinning, and it would be very helpful to have programmatic access to the pubkey fingerprint. The app currently has a huge and horrible mountain of platform- and backend-specific code that extracts the public key fingerprint from the TLS backend before

Re: libcurl build from source with openssl

2021-02-25 Thread Daniel Stenberg via curl-library
On Thu, 25 Feb 2021, Nir Azkiel via curl-library wrote: After reading more about it, it looks like libcurl is using deprecated functions like SSL_Close for example (Deprecated at OSX 10.13) Is there a way to make it work with 10.13 version? It might be deprecated, although that's the first

Re: libcurl build from source with openssl

2021-02-25 Thread Nir Azkiel via curl-library
rify_cert in libcurl.a(sectransp.c.o) > > 4945 _pkp_pin_peer_pubkey in libcurl.a(sectransp.c.o) > > 4946 _show_verbose_server_cert in libcurl.a(sectransp > After reading more about it, it looks like libcurl is using deprecated functions like SSL_Close for example (Deprecat

Re: libcurl build from source with openssl

2021-02-25 Thread Nir Azkiel via curl-library
Sorry for the top post, I hope this way is OK now (without the earlier posts). You helped a lot !! I didn't knew that gcc was sensitive about lib order. So now it works on Linux and macOs fail. I activated this flag CMAKE_USE_SECTRANSP=ON but I get linkage errors ., should I link a

Re: libcurl build from source with openssl

2021-02-24 Thread Ray Satiro via curl-library
On 2/24/2021 5:24 PM, Nir Azkiel via curl-library wrote: This is the cmake file for the libcurl:     include(ExternalProject)     message(STATUS "Building libcurl enabled")     set(CURL_FLAGS -DBUILD_CURL_EXE=OFF -DBUILD_SHARED_LIBS=OFF -DUSE_WIN32_LDAP=OFF -DHTTP_ONLY=ON -DCUR

Re: libcurl build from source with openssl

2021-02-24 Thread Nir Azkiel via curl-library
Sure, This is the cmake file for the libcurl: include(ExternalProject) message(STATUS "Building libcurl enabled") set(CURL_FLAGS -DBUILD_CURL_EXE=OFF -DBUILD_SHARED_LIBS=OFF -DUSE_WIN32_LDAP=OFF -DHTTP_ONLY=ON -DCURL_ZLIB=OFF -DCURL_DISABLE_CRYPTO_AUTH=ON -DCMAKE_USE_L

Re: libcurl build from source with openssl

2021-02-24 Thread Ray Satiro via curl-library
On 2/24/2021 9:40 AM, Nir Azkiel via curl-library wrote: I got libcurl downloaded and build, I would like to add HTTPS support but I keep getting linkage errors. I added this flag to libcurl Externalproject_Add -DCMAKE_USE_OPENSSL=ON also I added link to ssl like this set

libcurl build from source with openssl

2021-02-24 Thread Nir Azkiel via curl-library
Hi, I got libcurl downloaded and build, I would like to add HTTPS support but I keep getting linkage errors. I added this flag to libcurl Externalproject_Add -DCMAKE_USE_OPENSSL=ON also I added link to ssl like this set(OPENSSL_USE_STATIC_LIBS TRUE) find_package(OpenSSL REQUIRED

libcurl is at ~30K + download buffer for dynamic memory

2021-01-21 Thread Daniel Stenberg via curl-library
Hi, curl now uses about 30K of dynamic memory for downloading a large HTTP file, plus the size of the download buffer. I've been digging around and analyzed memory use and struct sizes in curl over the last few years. You can get all the details on my blog post about it:

Re: Does libcurl support all streaming relative protocols, e.g., RTP/RTCP/RTSP and so on?

2020-12-31 Thread Hongyi Zhao via curl-library
e: > >> > >>> I only see the RTSP protocol is supported by libcurl, but for streaming > >>> media relative protocols, I also learned that the RTP/RTCP/RTSP protocols > >>> are all needed to co-operate for complete the whole job. > >> > >>

Re: Does libcurl support all streaming relative protocols, e.g., RTP/RTCP/RTSP and so on?

2020-12-31 Thread René Berber via curl-library
On 12/31/2020 6:14 PM, Hongyi Zhao via curl-library wrote: On Fri, Jan 1, 2021 at 7:51 AM Daniel Stenberg wrote: On Thu, 31 Dec 2020, Hongyi Zhao via curl-library wrote: I only see the RTSP protocol is supported by libcurl, but for streaming media relative protocols, I also learned

Re: Does libcurl support all streaming relative protocols, e.g., RTP/RTCP/RTSP and so on?

2020-12-31 Thread Hongyi Zhao via curl-library
On Fri, Jan 1, 2021 at 7:51 AM Daniel Stenberg wrote: > > On Thu, 31 Dec 2020, Hongyi Zhao via curl-library wrote: > > > I only see the RTSP protocol is supported by libcurl, but for streaming > > media relative protocols, I also learned that the RTP/RTCP/RTSP protocols >

Re: Does libcurl support all streaming relative protocols, e.g., RTP/RTCP/RTSP and so on?

2020-12-31 Thread Daniel Stenberg via curl-library
On Thu, 31 Dec 2020, Hongyi Zhao via curl-library wrote: I only see the RTSP protocol is supported by libcurl, but for streaming media relative protocols, I also learned that the RTP/RTCP/RTSP protocols are all needed to co-operate for complete the whole job. Out of those three, only RTSP

Does libcurl support all streaming relative protocols, e.g., RTP/RTCP/RTSP and so on?

2020-12-31 Thread Hongyi Zhao via curl-library
, SMTPS, TELNET and TFTP. libcurl offers a myriad of powerful features. Currently, I want to implement real-time stream media transfer based with libcurl for on-line monitoring of the remote driverless vehicle. I'm not sure whether the protocol and capability shipped in libcurl can do such a job for me

Re: libcurl read-like interface

2020-12-27 Thread Daniel Stenberg via curl-library
On Sun, 27 Dec 2020, XSLT2.0 via curl-library wrote: It is still scary to read in libcurl's documentation that when pausing http/2 libcurl might buffer up to 32MB of data for each transfer! Is there a "SETOPT" to control that? No, but we could make one. The issue is of course tha

Re: libcurl read-like interface

2020-12-27 Thread XSLT2.0 via curl-library
> We discourage users from using curl_easy_recv() > for doing any protocol that libcurl implements natively.  Ok, found it! The solution was obvious, sorry for all the blurps: curl_easy_pause() Inside the multi_perform/callbacks caller's loop, it allows better control over the "tr

Fwd: libcurl read-like interface

2020-12-26 Thread XSLT2.0 via curl-library
l the tests I have done -a few kilobytes-, hence this is close enough to loop inversion, even if libcurl is still technically owning the transfer loop. I can't explain why in http/2 it is very unreasonable! ./fcurl_transfer https://www.youtube.com/s/player/5dd3f3b2/player_ias.vflset/en_US/base.j

Re: libcurl read-like interface

2020-12-26 Thread Daniel Stenberg via curl-library
On Sat, 26 Dec 2020, XSLT2.0 via curl-library wrote: You are not responding to the point about "control inversion" though. What am I to respond about that? It could use some polish to avoid risking the situation you describe. -- / daniel.haxx.se | Commercial curl support up to 24x7 is

Re: libcurl read-like interface

2020-12-26 Thread XSLT2.0 via curl-library
on I am using is running the curl transfers in separate threads, and blocking the callback with a semaphore when I have all the data I need to satisfy reads. That makes a complex code with thread communication, locking, atomic counters, etc... much prone to errors an bugs. That is because the "tr

Re: libcurl read-like interface

2020-12-26 Thread Daniel Stenberg via curl-library
On Sat, 26 Dec 2020, XSLT2.0 via curl-library wrote: Fortunately http/1.1 is simple enough, especially when downloading the "body" of a file (with Connection: close), you must admit that redoing what libcurl does here amounts to nothing. This could be one of the biggest understate

Re: libcurl read-like interface

2020-12-26 Thread XSLT2.0 via curl-library
BIO stack with possible proxies in the mix.   http/1.1 GET is simple and in this case curl_easy_read() performs about the same task as curl_easy_recv(). It still makes sense if we want gzip/http 2, libcurl can simply add "filters" on top of the stack before exposing curl_easy_

Re: libcurl read-like interface

2020-12-26 Thread XSLT2.0 via curl-library
True and duly noted Daniel! Fortunately http/1.1 is simple enough, especially when downloading the "body" of a file (with Connection: close), you must admit that redoing what libcurl does here amounts to nothing. I am even glad with avoiding a "full relay copy" (every bit

Re: libcurl read-like interface

2020-12-25 Thread Daniel Stenberg via curl-library
age users from using curl_easy_recv() for doing any protocol that libcurl implements natively. Mostly because you need to pretty much re-implement everything yourself instead of letting libcurl do what it was designed to do. -- / daniel.haxx.se | Commercial curl support up to 24x7 is available!

Re: libcurl read-like interface

2020-12-25 Thread XSLT2.0 via curl-library
tion last year about any interest for a "read-like" interface. But since this attempt didn't seem to have sparkled many interest so far, not speaking for the libcurl team, but I am not under the impression that this "new API" will be added anytime soon! I had already pointed

Re: libcurl read-like interface

2020-12-25 Thread Jiahao XU via curl-library
io_uring might take us into a future where no syscall only memory writes has to be made from within libcurl and the user of libcurl can take control of submitting io requests to the kernel completely. It is a technology that might requires libcurl to be redesigned as it is a new io request

Re: libcurl read-like interface

2020-12-25 Thread Jiahao XU via curl-library
tps://aka.ms/o0ukef> From: curl-library on behalf of XSLT2.0 via curl-library Sent: Saturday, December 26, 2020 4:17:51 AM To: curl-library@cool.haxx.se Cc: XSLT2.0 Subject: libcurl read-like interface (Continued) After some homework, I would comment on "re

libcurl read-like interface

2020-12-25 Thread XSLT2.0 via curl-library
o memory pool, and from there to caller's buffer, plus additional copies triggered by realloc and memory shifts when data is consumed! Looking at OpenSSL BIO stack/filter architecture, this "read like potential inefficiency" is because libcurl is already running all the stack/filte

re: libcurl read-like interface

2020-12-18 Thread XSLT2.0 via curl-library
> How much extra time does that memcpy() relay > a 10GB transfer. Then we measure wall clock time. Summary (Rasp. Pi 4 - TLS) : Real ("Elapsed") +2.9 s [ +2.74 s "User" / +0.16 "Kernel" ] Full results, test methodology and comment: 

re: libcurl read-like interface

2020-12-17 Thread XSLT2.0 via curl-library
hy measure user time? We want to see the performance inpact as a whole, right? How much extra time does that memcpy() relay a 10GB transfer. Then we measure wall clock time. I beg to differ Daniel: that is the wrong tool... if the goal is to measure the impact of a change on libcurl + callback (like adding

Re: Fwd: libcurl read-like interface

2020-12-16 Thread Daniel Stenberg via curl-library
ot; test has any interest as a "reverse" benchmark? I'm not sure it adds much value to create another synthetic benchmark. So yes, the valid question would be: how many users of libcurl need something much more sophisticated than "writing to a file" with some "de

Fwd: libcurl read-like interface

2020-12-16 Thread XSLT2.0 via curl-library
expected to be "equal or worse" than my simple test. >You mean, the one that ends "Yet we should not >pass up our opportunities in that critical 3%"? That is a excellent question! Does libcurl need "caller provided buffers" (aka "zero/less copies")? T

Re: libcurl read-llike interface.

2020-12-14 Thread Daniel Stenberg via curl-library
e of fcurl is ideal) that uses "multi-socket" to get a taste and try to understand the subtle differences between multi-select and multi-socket! I don't know. I don't keep any records of what libcurl-using applications use what libcurl API. The primary difference is that multi

Re: libcurl read-llike interface.

2020-12-14 Thread XSLT2.0 via curl-library
Raspberry Pi where the bottleneck is NOT the network (ethernet + fiber connection) but in fact CPU used for crypto (eg SSL/TLS), saving cycles makes things somehow quicker. For example, I doubled "efficiency" (as seen with top/htop) on my fuse driver that uses libcurl, by changing write buffer

Re: libcurl read-llike interface.

2020-12-14 Thread Tomalak Geret'kal via curl-library
On 14/12/2020 16:35, Daniel Stenberg via curl-library wrote: > On Mon, 14 Dec 2020, XSLT2.0 via curl-library wrote:  > >> It is also pretty un-optimised: the write callback could >> be called multiple times in a multi_perform cycle. > > There's this saying about premature optimization... You

Re: libcurl read-llike interface.

2020-12-14 Thread Daniel Stenberg via curl-library
zero-copy libcurl (a topic much discussed!) I don't think it will help with that... Vocabulary (as often)! It depends on what you call "zero-copy" here. Sure, zero-copy usually means fewer-copies and that's fine, but I still don't think the fcurl API is the key to solving that. But I'm

libcurl read-llike interface.

2020-12-14 Thread XSLT2.0 via curl-library
un-optimised: the write callback could be called multiple times in a multi_perform cycle. This is correctly handled, but can lead to unknown memory consumption (bad if you work with limited memory) and multiple copies of the same data due the realloc technique! > -2) A step towards zero-copy li

Re: libcurl read-llike interface.

2020-12-13 Thread Daniel Stenberg via curl-library
On Sun, 13 Dec 2020, XSLT2.0 via curl-library wrote: I saw on the 2020 poll an item about adding to libcurl a "read-like interface" (instead of callbacks). I didn't find any detail on that feature. Well, it was a question in a poll about it. It doesn't have to mean that it actua

libcurl read-llike interface.

2020-12-13 Thread XSLT2.0 via curl-library
Hello, I saw on the 2020 poll an item about adding to libcurl a "read-like interface" (instead of callbacks). I didn't find any detail on that feature. Would you have a link or some reading that could help me understand what the benefits would be? From the most obvious to the more

[SECURITY ADVISORY] libcurl: FTP wildcard stack overflow

2020-12-08 Thread Daniel Stenberg via curl-library
FTP wildcard stack overflow === Project curl Security Advisory, December 9th 2020 - [Permalink](https://curl.se/docs/CVE-2020-8285.html) VULNERABILITY - libcurl offers a wildcard matching functionality, which allows a callback (set

Re: libcurl SSL session reuse & connection reuse

2020-11-13 Thread Daniel Stenberg via curl-library
as there is no housekeeping after the 118s, meaning that we will be connected to the server until next operation. If nothing closes the connection, it will remain open indeed. But libcurl will never use it again. So for this reasons something like easy_close_current_connection(curl obj) would be perfect

Re: libcurl SSL session reuse & connection reuse

2020-11-13 Thread Alen Loncaric via curl-library
Dear, yes this options are valid, but they do not solve every scenario. Consider the following: Having a long poll loop in on thread (connection reuse, ssl ticket reuse) Another thread doing some log pumping to a rest API (can send 1 or more requests) - should reuse only SSL ticket id and create

Re: libcurl SSL session reuse & connection reuse

2020-11-13 Thread Daniel Stenberg via curl-library
On Fri, 13 Nov 2020, Alen Loncaric via curl-library wrote: The problem is that for this rest calls connections are also added to connection cache. Is there a way to prevent this ? You can set CURLOPT_FRESH_CONNECT to prevent a new transfer from re-using any existing previous connection and

libcurl SSL session reuse & connection reuse

2020-11-12 Thread Alen Loncaric via curl-library
Hi all, first off all thank you for all the good work! I have a shared object that holds SSL session ticket and connection cache for our long poll operation. Also while our long poll is alive we have some rest calls that are using also the same cache for ssl ticket reuse. The problem is that for

Re: About libcurl- Help, please

2020-11-12 Thread Daniel Stenberg via curl-library
nded. 4. "I cannot get this command to work well" doesn't explain what you want nor what happens. Many times the actual HTTP request works fine but the end results is not what you want, even if curl works exactly the way it is supposed to. We need specifics! 5. You didn't tell us whic

Re: About libcurl- Help, please

2020-11-12 Thread m brandenberg via curl-library
the libcurl commands? Use the '--libcurl' option. The command will write the C code for you. m -- Monty Brandenberg, Software Engineer MCB, Inc. mcb...@panix.com P.O. Box 425292 mcb...@pobox.com

About libcurl- Help, please

2020-11-12 Thread Osvaldo Pérez García via curl-library
Hello: I'm starting to learn how to use libcurl to automate some of the tasks we do using the curl command. I am programming in C for FreeBSD. So far I have been progressing, but I cannot get this command to work well in the program: curl -X GET --header 'Accept: application / json' --header

Re: Performance issue with libcurl 7.73

2020-10-30 Thread Ray Satiro via curl-library
On 10/29/2020 7:57 PM, Jeffrey McKay via curl-library wrote: Recently we did have some problem with logging in to some servers, and I determined that the issue could be fixed by updating to libcurl 7.71 (not sure why). However I ran into some other problem with this version, where for some

Performance issue with libcurl 7.73

2020-10-29 Thread Jeffrey McKay via curl-library
Hello, I maintain a Windows application (written in VC 2017) that basically uploads thousands of email messages into Microsoft Exchange/Ofice 365, via their EWS protocol. I have been successfully using libcurl 7.64 for a while. Recently we did have some problem with logging in to some servers

Re: libcurl hang (v2)

2020-10-22 Thread Daniel Stenberg via curl-library
On Wed, 21 Oct 2020, MS via curl-library wrote: I have discovered the problem only occurs when using libevent2 to drive the multi interface. There is no issue using the easy interface directly, nor using `multi_poll'. ... then the problem is probably in your event-based logic! -- /

Re: libcurl hang (v2)

2020-10-21 Thread MS via curl-library
0 at 03:50:00PM +0100, MS via curl-library wrote: Gisle: thank you for your response. Let's focus on the different behaviour with IPv4: Using libcurl in an application, without firewall operating, attempting to connect to bbc.co.uk, I obtain a hang with *   Trying 151.101.64.81:80... the on

Re: libcurl hang (v2)

2020-10-21 Thread Dan Fandrich via curl-library
On Wed, Oct 21, 2020 at 03:50:00PM +0100, MS via curl-library wrote: > Gisle: thank you for your response. > > > Let's focus on the different behaviour with IPv4: > > Using libcurl in an application, without firewall > operating, attempting to connect to bbc.co.

libcurl hang (v2)

2020-10-21 Thread MS via curl-library
Gisle: thank you for your response. Let's focus on the different behaviour with IPv4: Using libcurl in an application, without firewall operating, attempting to connect to bbc.co.uk, I obtain a hang with *   Trying 151.101.64.81:80... the only output written. Using curl.exe I obtain

Re: libcurl hang

2020-10-21 Thread Gisle Vanem via curl-library
MS wrote: 1) Using libcurl without a proxy, the verbose options shows "Trying ip address" and nothing else. If restricted to IPv4 it shows only one line. If IPv6 is permitted, I get a first response that network is not reachable and then a second response "Trying ..."

libcurl hang

2020-10-21 Thread MS via curl-library
I have a hang that I am attempting to resolve. 1) Using libcurl without a proxy, the verbose options shows "Trying ip address" and nothing else. If restricted to IPv4 it shows only one line. If IPv6 is permitted, I get a first response that network is not reachable and the

  1   2   3   4   5   6   7   8   9   10   >