Den 18-05-2011 14:52, Johannes Pfau skrev:
Jonas Drewsen wrote:
On 18/05/11 10.09, Johannes Pfau wrote:
jdrewsen wrote:
Please see comments below.
Den 17-05-2011 16:42, Andrei Alexandrescu skrev:
Thanks for the response! A few more answers and comments within
(everything deleted counts as "sounds great").
On 5/17/11 3:50 AM, Jonas Drewsen wrote:
14. Isn't the max redirect configurable via a parameter?
Yes. It is called Http.followLocation from libcurls followLocation
option. I will rename it to maxRedirections for clearity.
You mention this about many higher-level functions: "Maximum
redirections are set to 10." Then I'm thinking, if it's an
important parameter, make it a defaulted parameter for the
function.
I guess I should just remove that comment because other defaults has
been selected as well e.g. timeouts.
16. Documentation should point to descriptions of the HTTP
methods wrapped (e.g. "post", "del" etc).
Do you mean point to the relevant RFC?
Yah, or a good tutorial. (I didn't know DEL existed!)
Well DEL is actually called DELETE in the HTTP RFC. But delete is a
reserved word i D so I used del() instead. delete() wouldn't work in
following code in think:
with(auto http = ...) {
delete(...);
}
22. byLine/byChunk should not expose a string or generally data
that can be shared safely by the client. That's because it would
need to create a new string with each iteration, which is very
inefficient. Instead, they should expose char[]/ubyte[] just like
File.byLine does, and reuse the buffer with each call.
Essentially byLine/byChunk should be near-zero-overhead means to
transfer and inspect arbitrarily large streams.
byLine/byChunk is currenly only available for the async methods
which is implemented using message passing. This means that for
passing the message a copy has to be made because a message is
by-value or immutable data. Is there another way for me to pass
the message without doing a copy... some kind of move semantic?
A library element is planned but not on the short list yet. You can
work around that by internally doing a cast. As long as you observe
the commitment that buffers are not accessed simultaneously by more
than one thread you're well within defined behavior.
ok nice to know.
One more suggestion - you may want to also provide classic boring
synchronous read/write methods that take a user-provided buffer.
I'm sure people could use such e.g. when they want to stream data
inside their own threads, or even in the main thread if they don't
mind blocking.
This would mean hooking into libcurl and selecting on its sockets.
Totally doable.
I'm not sure if it'd be that easy. Using select to implement a
blocking api is possible, but select is only used to wait until data
is ready, reading and writing the data is still done by curl. And as
curl does not allow you to supply the data buffer, having
user-provided buffers is afaik impossible.
_Doable_ - not easy :)
Select will wait for data to be ready and ask curl to handle the data
chunk. Curl in turn calls back to a registered callback handler with
the data read. That handler fills the buffer provided by the user. If
not enough data has been receive an new select is performed until the
requested amount of data is read. Then the blocking method can return.
I think it's not necessary to do multiple calls if the first didn't
return enough data. Most read calls are only guaranteed to return a
maximum of data, less is always possible. It's a bigger problem if
the user supplied buffer is smaller than the curl buffer. And you
always end up copying data. So it might be better
to just return a reference to the buffer allocated by curl, or just
forget about this api, curl just doesn't work well this way.
Yeah. That's part of the reason why I'm not interested in implementing it.
But I would rather stop here feature wise and wrap
this thing up. I would like to focus my efforts on a candidate for
std.net.event/reactor/proactor module.
Have you already started working on std.net.event? Do you plan to
base that on libev / libevent?
I'll finish the curl wrapper before starting on it.
I have had a look at libev/event before and they are very nice libs.
But I think I'll have to implement it from scratch for two reasons:
1, AFAIK we cannot include code in std phobos that is not boost
licensed or a license as liberal. libev for example requires you to
distribute the license with your software.
That could indeed be a problem. I think libev/libevent even have the
same license. Probably it's really best to implement it from scratch,
but it will be quite some work. I'd still have a look at the libev
documentation though, it contains some details about OS quirks. And try
to avoid Linux AIO ;-)
2, It introduces a dependency to an external project in phobos.
Currently no dependencies are present. This makes phobos very easy to
install and use out of the box.
The optimal solution to these problems from my point of view would be
to get that "much discussed" package tool in place soon (CPAN,apt
alike).
Heck, now I think about it maybe I should do that before any std.net
stuff. Let's see how the wind blows - many interesting projects :)
I've been trying to get familiar with libev lately, so I have
bindings for livev 3.9 / 4.04 (combined, version can be selected
with version statements):
http://dl.dropbox.com/u/24218791/d/src/libev.html (comments in the c
header have been turned into doc comments. The binding is partially
based on Leandro Lucarella bindings:
http://git.llucax.com.ar/?p=software/ev.d.git)
Very nice. I've also stumbled upon some other async D libs out there
that could be a good starting point.
It seems that Leandro has both bindings and a wrapper in place. And a
wrapper is really nice to have to make it more D'ish.
Yes, I have a higher level wrapper for libev as well, but the new
'shared' stuff made it a lot more complicated to get that right.
/Jonas
