On Wed, 22 May 2013 21:14:55 +0100 Ben Thomas wrote: >On Wed, May 22, 2013 at 6:49 PM, Daniel Stenberg <daniel_at_haxx.se> wrote: >> A) use really larger buffers, like several hundred KBs at least >> >> and >> >> B) use a sliding buffer technique instead of letting the existing buffer >> drain completely before it gets refilled >> > So is it possible for me to do A and B using cURL?
You can do A) by compiling a custom libcurl configured with something like CPPFLAGS=-DCURL_MAX_WRITE_SIZE=1048576 That will change the default 16 KiB buffer size to 1 MiB, which will kill your memory footprint but should speed up your transfers. libcurl isn't expecting such a large buffer so it isn't particularly careful about using it efficiently, but I've tried setting it as high as 32 MiB and it still passes the test suite (and takes 132 MiB of RAM to download a 200 byte file over HTTP!) >>> Dan ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
