ou could get the cookies from the server with “write header call back” [1] and
set the cookies in future requests with CURLOPT_COOKIE [2].
[1] https://curl.haxx.se/libcurl/c/CURLOPT_HEADERFUNCTION.html
[2] https://curl.haxx.se/libcurl/c/CURLOPT_COOKIE.html
Same could be done with curl tool (not libcu
e is an event on the sockets / timeout.
Please look at the code example on curl_multi_wait, it might help you a lot.
[1] https://curl.haxx.se/libcurl/c/curl_multi_info_read.html
[2] https://curl.haxx.se/libcurl/c/curl_multi_wait.html
HTH
Yehezkel Horowitz
--
>Any ideas? I'm guessing it's something small i'm missing but for the
>life of me I can't figure out what it is.
>
>Cyd
Just to make sure - did you run 'configure' before trying to build with 'make'?
Yehezkel Horowitz
. Look in
http://curl.haxx.se/libcurl/c/curl_share_setopt.html
If you go for the multi interface, you will not need it, it is done by default.
PS - you might use timeouts functionality of curl to limit the time you are
waiting for the server.
HTH
Ye
> Here's my suggested patch. It makes the test program work again (and all
> other tests continue to run). I've also made a test case out of it
> separately, to avoid future regressions on this.
Does this bug affect only users that use curl_multi API and set the binary
data after adding the ea
.
Another issue - when you go to libcurl section, the upper menu is too long and
take 2 lines, which make the UX not so good.
Anyhow, thanks Daniel for taking time to improve the site.
Regards,
Yehezkel Horowitz
---
List a
>Yes, I too think that server push and HTTP/2 streams in general will only grow
>in importance and popularity going forward. But we also don't need to invent
>everything or fix every possible future problem now, we just need to lay the
>foundation good enough and then we continue improving and p
Hi Daniel
> I've started out this week working on HTTP/2 multiplexing for libcurl. My
> idea is that when using the multi interface and adding easy handles, all
> handles that identify the same "origin" (host + port + protocol combination
> really) will use the same single connection and do "re
>I'm currently using CURL_EASY_PERFORM to send data to a remote URL. I notice
>that in my current configuration it appears to be behaving synchronously (i.e.
>waiting until completion of function). Since at times I may have several of
>these calls executing within a short duration I'd like to ma
rt a new connection with FRESH_CONNECT option.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
e your application forget this info)
authentication method
Once you have this info - you can use curl options to do the authentication for
you.
Relevant curl options are:
CURLOPT_HTTPAUTH
CURLOPT_USERPWD or CURLOPT_USERNAME + CURLOPT_PASSWORD
HTH
Yehezkel Ho
the environment variable
KRB5CCNAME to point to the correct cache before running curl.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
> Each thread in my client application creates a CURL handle using
> curl_easy_init, sends a request to the server and once the response is
> received, invokes curl_easy_cleanup on the handle. So, the connection is
> closed by the thread owning the CURL handle on the client side.
Why you call
send FIN/RST?
Did you set FORBID_REUSE option?
You can also try adding some more IP addresses to the server (DNS round-robin)
and get a bigger range of IP:port.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/li
opinion and personally I didn't have any real problem in the
documentation area of the libcurl project (as all needed info is in the
site/mailing list all you need to do is to search for it).
HTH
Yehezkel Horowitz
---
List admin
and then we can burry fbopenssl once and for
>all.
AFAIK, GSS-API is implemented in krb5 library and libcurl use this API (and not
Kerberos API), but for servers who enforce support of SPNEGO - you must use
SPNEGO API (which the only open-source implementation I know is fbopenssl).
BTW - what
>As reported at http://bugs.debian.org/749171, 1080 is the default port for
>socks (as defined by IANA), but http proxies use 80 as default (which is what
>e.g.
wget does).
I had the impression that default port of http proxies is 8080, but I might be
wrong...
Yehezkel
>> I would like to have a "CURLOPT_BUFFER" option, so I can instruct
>> libcurl to copy the data there directly. What are your thoughts about
>> adding it/accepting a patch?
>Since libcurl may download an infintely large amount of data, just pointing
>out a single buffer is not enough. It would
revent the re-use).
Look at "Sharing Data Between Easy Handles" in
http://curl.haxx.se/libcurl/c/libcurl-tutorial.html for more info.
You can also look at section " Multiple Transfers Using the multi Interface"
(same URL) fo
e connections ( = manage the
re-use by yourself).
Regards,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
fully and which failed).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
ter to whatever you want passed in to your
Regards,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
etopt.html#CURLOPTDEBUGFUNCTION
Regards,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
on get a "timeout" event.
Hope this help you to understand the process.
PS - avoiding calling curl_easy_cleanup will just result in memory leak...
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
(which we should announce and report)?
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
e able to "rewind" in case
curl ask you for with CURLOPT_IOCTLFUNCTION or CURLOPT_SEEKFUNCTION.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
d you please tell me how can my application recognize for retry in this
>case?
You should use CURLOPT_IOCTLFUNCTION and you will be called with
CURLIOCMD_RESTARTREAD.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/lis
form.
-- Original ------
From: "Yehezkel
Horowitz";mailto:horow...@checkpoint.com>>;
Date: Thu, Dec 5, 2013 06:22 PM
To: "libcurl
development"mailto:curl-library@cool.haxx.se>>;
Subject: RE: slow uploading speed for single task on osx
>
wrong usage, as the calls should be done according to events
(socket/timeout).
Please give us more details when you call
curl_multi_perform/curl_multi_socket_action from your code (what is the trigger
to those calls), and when you call it with CURL_SOCKET_TIMEOUT
> // THIS IS WHAT I WANT TO CALL
> // memberfunction:
> Void onFinishedMemberCallback(...){
> //use collected data, callback owner and cleanup
> }
> // THIS IS ALL I CAN CALL
> Static Void onFinishedOrOtherEventCallback(..., CURL* easy
this is an open-source project - you are welcome to contribute your
patches (which you did - thanks), but I think you should take care with your
statements in such a list.
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list
T_SSL_VERIFYHOST are your friends.
But they have a secure default values (validate the server certificate).
Can you show us how you are using curl (and the verbose logs)?
If you are using the curl (not libcurl), you should NOT have -k option in you
command-line.
Regards,
Y
rtificate (this is the
name curl should verify to match the certificate).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
in on the same easy handle (or create a new one
and configure it with only IPv4 addresses).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
ly SPNEGO
authentication and not direct NTLM/Kerberos (don't have details how it was
configured).
You also assume that libcurl run with recent (< 4 years) Kerberos code, which
is not always the truth.
Yehezkel Horowitz
---
libcurl/c/curl_easy_setopt.html#CURLOPTHTTPVERSION).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
url_easy_setopt.html#CURLOPTDEBUGFUNCTION)
output of your attempts so we can better understand your exact problem.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
>When a single transfer is completed in the multi handle, we can remove the
>easy handle and set new options to start it again. But how can I know is the
>easy handle connected or idle?
You should use curl_multi_info_read to get messages about easy handles who
finished their transfer.
curl_multi_select which probably do the job
for you.
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Answer to a little bit old question...
> I'm seeing strange behavior where requests are _much_ faster when routed
through a proxy (in this case the Fiddler diagnostic tool). Note that this is
not a caching proxy, and all the requests are POSTs anyway, so I don't think
that is the explanation. I
tp://curl.haxx.se/) to the security advisory is broken
(leads to http://curl.haxx.se/docs/adv_201302026.html), can you please fix this?
Regards,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
match your needs).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
I think it will only handle
GET requests).
HTH
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
ternally uses a poll/select call on the active sockets and
>indirectly monitors the sockets in the handle.
Thank, now it makes sense.
My bad, sorry (lesson I learned - never trust the function name to tell the
whole story, instead - read the manual…).
This could decrease the performance of libcurl, which is one of the killing
features it has.
Just my thoughts - hope I didn't missed anything.
Regards,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/c
just call to curl_easy_reset(curl_handle), and re-use the same
exact handle to perform the 2nd transfer.
> curl_easy_perform(curl_handle2)
This is a different time (just after the first transfer was ended), but you
will have 2 curl handles allocated on your stack - no problem with that.
>If you disable libssh2 by using the configure option --without-libssh2, then
>that should turn off SCP and SFTP.
Thanks Nick.
Yehezkel
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/
ition, I didn't found a way to disable SCP (--disable-scp), did I missed
something?
Thanks,
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
First - thanks for your quick reply.
>> I wonder if there is an easy way to compile libcurl into different
>> SONAME like libmycurl.so?
>Sure. If you check the lib/Makefile you can see that VERSIONINFO is being set
>(to an argument that is passed along to libtool). You can set your own value
>
patched library, while others will use
the regular library.
I'm quite sure there is such way (changing some configure lines) but I failed
to find it.
Thanks in advanced,
Yehezkel Horowitz
Check Point Software Technologies Ltd.
---
chine).
Still you need a library that will have the **server side** logic. There are
many projects that do this, you can see libmicrohttpd
(http://www.gnu.org/software/libmicrohttpd/) as an example.
HTH
Yehezkel Horowitz
--
l best answer your requirements).
Yehezkel Horowitz
---
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
t in the curl
spirit.
I don't think we should even consider SONAME change for this issue (I don't
think the behavior of checking the existence of CN in the subject, is someone
in production environments really need).
Yehezkel Horowitz
---
I saw nobody agreeing to that.
Even if they will agree to do the change, it will take time till their users
(which version?) will upgrade, so in the meanwhile we will have curl-php users
that will still use TRUE (==1) as an argument to 'CURLOPT_SSL_VERIFYHOST'.
Yehezkel Horowitz
>cURL is also mentioned in the FAQ at
>https://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html
This is the quote from the FAQ
>Q: How do I use cURL securely?
>A: CURLOPT_SSL_VERIFYPEER must be set to TRUE, CURLOPT_SSL_VERIFYHOST must be
>left to its default value or set to 2. Any
> I wonder how I can know if curl convey the form content to the web site and
> what it posts.
> …My script failed to post. Any suggestion is welcome. Thanks.
The very first step you can do is to use the CURLOPT_DEBUGFUNCTION to see the
whole transaction.
Yehezkel
see only the "real" data.
In case you are acting as a proxy and pass this data you should either remove
the "Transfer-Encoding: chunked" header (in your write_header callback) OR
kindly ask libcurl not to decode the stream with CURLO
> Is there a ssl specific data structure I can use for this scenario ? How do I
> get it from the CURL* handle ?
You can consider using CURLOPT_PRIVATE & CURLINFO_PRIVATE for this.
HTH
Yehezkel Horowitz
---
List a
I think these lines are the problem with your approach
> while (still_running){
> curl_multi_perform(multi_handle, &still_running);
> usleep(10);
> }
You call curl to progress whenever you want (why you just sleep 100K?!),
instead of when curl needs to
>it returns CURLE_OK although the url i want to upload to is not allowed for
>upload.
>CURLINFO_RESPONSE_CODE returns 403
>CURLE_HTTP_RETURNED_ERROR or CURLE_UPLOAD_FAILED would be expected.
>Any suggestions for that?
CURLE_OK means that the HTTP transaction completed successfully and you got
> Thanks everyone for your feedback. Just going for the reject-the-invalid case
> was also very simple, so this is what I just did and a change has been pushed.
Adding one more reason to use the strict option - Just imagine all the support
mails that will claim curl to have a bug in case curl w
> I'm using the curl_easy_* API to do a simple HTTP GET. The response is long
> enough to be split up into several calls to the callback CURL_WRITEFUNCTION.
> I wonder how I know when all the data has been received? When can I tell,
> with certainty, that no more calls will be made to my callbac
> Right, but the current fixed-size 32K (16+16) allocation could easily be
converted into a more dynamic way that either would automatically grow
according to some rule or it could be possible to set with an option.
> A first and simple zero-copy implementation for libcurl would still have to
p
>> It was removed from the list a few months ago, presumably due to lack of
>> interest.
> Indeed.
> if I would reconsider it, someone would have to show a significant
> performance boost with the use of something like that.
Since such approach will require dynamic buffer allocation, I think
>> Limiting outstanding requests, implementing request queues in libcURL:
>This is also frequently missed by users and will indeed make it even easier
for libcurl users to write programs without duplicating logic from other
programs to get this very common pattern implemented. I'm all for it!
I
>> Are you planning to develop this mode?
>No, I am not. I have no need for it, nobody is paying for it and I consider a
>lot of my other ideas and tasks more important.
>> Is there a way I can help with this development effort?
> Yes, and I would probably say it is even required. Someone needs
A week has passed with no reply, so I'll ask again:
Are you planning to develop this mode?
Is there a way I can help with this development effort?
>> Finally, I would be glad to know if you think this mode is worth development
>> effort, and if you think curl users (other than me) will be happy
>> Finally, I would be glad to know if you think this mode is worth development
>> effort, and if you think curl users (other than me) will be happy to use it.
> I honestly don't know, but I would suspect so. Lots of people already have
> the problem of separating the "fake" headers from the "re
> I can easily come up with one case right now and I suspect there are others
> if we just give it some more thoughts.
> Assume your application enables authentication with a specific subset of
authentication types. When the 401 response comes libcurl doesn't know if it
is the final or not, so
>I agree that it could make sense with a mode that makes libcurl hold the
headers for a while and if it can deal with them itselves for auth reasons etc
it would hide them completely from the application. That same logic (or
similar) could also hide headers for 30X redirects as it has the simil
o need to get all this information;
>all this could and should be handled internally inside libcurl, since the
>username and password needs to be configured in advance.
What do you think?
Yehezkel Horowitz
Check Point Softwa
>Hello,
>I recently started moving from synchronous curl easy handle approach to the
>asynchronous way using curl multi handle.
> But I am stuck with some questions before finally making a choice between
>curl multi perform and curl socket action.
>My questions are as below:
>1) Natu
>> I want to use curl_multi interface along with epoll (can't use select since
>> I want to perform much more than 1024 requests in parallel).
>>
>> My problem is that I can't get the "new" sockets that I need to add to the
>> epoll_ctl, just the whole set of fds (by using curl_multi_fdset).
>Ri
dle from associated multi
handle it get lost all its open connections :-(
How should I proceed? Is there an option to get my goal (use more than 1024
easy handles in parallel with curl_multi interface)?
Take into account - my goal is to have ~20K connections opened in parallel.
Thanks in adv
74 matches
Mail list logo
