On Sun, 27 Dec 2020, XSLT2.0 via curl-library wrote:
It is still scary to read in libcurl's documentation that when pausing
http/2 libcurl might buffer up to 32MB of data for each transfer!
Is there a "SETOPT" to control that?
No, but we could make one. The issue is of course that the 32MB
> We discourage users from using curl_easy_recv()
> for doing any protocol that libcurl implements natively.
Ok, found it!
The solution was obvious, sorry for all the blurps: curl_easy_pause()
Inside the multi_perform/callbacks caller's loop, it allows better
control over the "transfer loop"
"Loop inversion"
I have fixed fcurl for the read scenario (made a PR).
Made a basic fcurl_read/write_to_stdout program with fcurl.
(extract)
while(!fcurl_eof(fcurl)) {
sz = fcurl_read(buf, 1, BUFSIZE, fcurl);
if (sz > 0) {
wsz = write(STDOUT_FILENO, buf, sz);
On Sat, 26 Dec 2020, XSLT2.0 via curl-library wrote:
You are not responding to the point about "control inversion" though.
What am I to respond about that?
It could use some polish to avoid risking the situation you describe.
--
/ daniel.haxx.se
| Commercial curl support up to 24x7 is
I beg your pardon if I made you angry Daniel, understating the
"simplicity" of http!
You are not responding to the point about "control inversion" though.
Let me try to better explain what happens to a "read" with the current
behaviour. You know it since you have coded it yourself in "fcurl".
On Sat, 26 Dec 2020, XSLT2.0 via curl-library wrote:
Fortunately http/1.1 is simple enough, especially when downloading the
"body" of a file (with Connection: close), you must admit that redoing what
libcurl does here amounts to nothing.
This could be one of the biggest understatements I've
"Re-architecture"!
Instead of "only critique", here is a (possible) proposition to do both
"read-like" (efficiently) and the current stuff.
There should be a definition of what *semantically* is "read" or "write"
according to the request/use case.
The "do/doing/done" internal phase, should
True and duly noted Daniel!
Fortunately http/1.1 is simple enough, especially when downloading the
"body" of a file (with Connection: close), you must admit that redoing
what libcurl does here amounts to nothing.
I am even glad with avoiding a "full relay copy" (every bit of
efficiency helps
On Fri, 25 Dec 2020, XSLT2.0 via curl-library wrote:
So in fact, I'm glad that my "homework" made me discover the well hidden
*curl_easy_recv()*, I can make do something interesting from that, and still
use "classic callbacks" for WebAPI calls or the like.
We discourage users from using
Sorry Jiahao if I confused you!
To summarise what was said in "previous episodes":
- a "read-like" interface was considered 4 years ago and a quick PoC
(Proof of Concept) was made: fcurl (see https://github.com/curl/fcurl) -
last commit April 2016.
- there was a poll question last year about
model.
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Jiahao XU
Sent: Saturday, December 26, 2020 6:28:12 AM
To: libcurl development
Cc: XSLT2.0
Subject: Re: libcurl read-like interface
Just wondering, do you take io_uring into account when designing th
tps://aka.ms/o0ukef>
From: curl-library on behalf of XSLT2.0 via
curl-library
Sent: Saturday, December 26, 2020 4:17:51 AM
To: curl-library@cool.haxx.se
Cc: XSLT2.0
Subject: libcurl read-like interface
(Continued)
After some homework, I would comment on "re
(Continued)
After some homework, I would comment on "read".
First semantics:
fcurl_read implies the caller wants to read the "body", whatever that
means according to the protocol used.
There might still be needs for reading headers (and trailers?), and
could be accommodated either via current
> How much extra time does that memcpy() relay
> a 10GB transfer. Then we measure wall clock time.
Summary (Rasp. Pi 4 - TLS) : Real ("Elapsed") +2.9 s
[ +2.74 s "User" / +0.16 "Kernel" ]
Full results, test methodology and comment:
> Oh right, that's a fun way to try stuff.
My fisrt intention was to use fcurl with the very classic code (error
checking omitted):
while(!fcurl_eof(fcurl)) {
sz = fcurl_read( fcurl, buf, 1, BUF_SIZE);
write(1, buf, sz);
}
But since fcurl seems bugged, it was quicker to just add the
On Wed, 16 Dec 2020, XSLT2.0 via curl-library wrote:
Naive "reverse" benchmark adding an extra useless memcpy:
https://gitlab.com/BylonAkila/astreamfs/-/blob/master/benchmark/benchmark.c
Oh right, that's a fun way to try stuff.
Summary impact of a *single memcpy* on a typical HTTP GET: 11%
> Probably by doing a first prototype implementation
> which can be used to run benchmarks.
Naive "reverse" benchmark adding an extra useless memcpy:
https://gitlab.com/BylonAkila/astreamfs/-/blob/master/benchmark/benchmark.c
Summary impact of a *single memcpy* on a typical HTTP GET: 11% to
17 matches
Mail list logo
