On Thu, Jan 10, 2008 at 03:28:52PM -0800, John Meacham wrote: > On Thu, Jan 10, 2008 at 06:16:55PM -0500, David Roundy wrote: > > > Yes. I am actually working on adding persistant connections and > > > ByteString support to HTTP for this. I just wanted to get the quick > > > dirty and working version in there first and get some feedback. > > > > Great! That's exactly the news I wanted to hear! :) > > > > Have you taken followed the discussion we've been having about libwww and > > the internal API? I think that the combination of getUrls and waitForURL > > sounds like a good interface for pipelined downloads... although I notice > > you mention persistent connections but not pipelining. Any chance you'd be > > willing to take a shot at pipelining? > > Yeah, I meant pipelining actually. well, both.
Good. > I am not sure what the exact form of the API will take at the moment. I > was thinking something like: > > data RemoteConnection = .... > data Response = ... > data Token = ... > > -- create a connection object > newRemoteConnection :: IO RemoteConnection > > -- queue a request and what to do when you get the response. > addRequest :: RemoteConnection -> URL -> (Response -> IO ()) -> IO Token > > -- wait for a specific one to finish > waitForOne :: Token -> IO () > > -- wait for all to finish > waitForAll :: RemoteConnection -> IO () > > > which should be enough to provide the functionality of getUrls. Right. All I'd want to change beyond that API would be to add a cache of URL (or rather, String) -> Token, so that we can ask for the same URL several times without downloading it more than once (which is easy to layer on top of your API, and doesn't really belong in a general-purpose API). Actually, I have a couple of suggestions. I wonder what your Response data type will be. It would be nice to have the possibility of streaming the data directly to disk, which is how darcs currently operates. I'd also like to be able to stick the data in a lazy bytestring, so we could avoid using the disk as a buffer, but the former operation could be important, if we're speculatively downloading large files. Also, I presume you are thinking that waitForOne and waitForAll will throw exceptions if there's a problem with the download? -- David Roundy Department of Physics Oregon State University _______________________________________________ darcs-devel mailing list darcs-devel@darcs.net http://lists.osuosl.org/mailman/listinfo/darcs-devel
