On Thu, 10 Nov 2011, Kopparapu, Sangeeta (GE Healthcare, consultant) wrote:
My requirement is to download multiple small files (sizes starting from 1KB
and maximum of 200KB)) from a server (same) using HTTP protocol. I have used
libCURL easy interface to download files but performance is not as expected.
I also tried with curl_multi_perform (followed the example 10-at-a-time.c).
But no gain in performance. So, now, I wanted to use libCURL hiper
(curl_multi_socket* function) to achieve this.
The event-focused API is primarily meant to offer high performance and good
scalability when you raise the number of simultaneous transfers. If you're
doing your transfers in a serial, one by one, manner it doesn't offer any
major benefits. You didn't mention if you do your transfers in parallel or
not.
I have seen the examples that have come with libCURL source code to get
overview on how to use libCURL library. But they are all making use of 3rd
party libraries(libevent, libev..etc.) and so have some imact on overall
performance (which is also mentioned in
http://curl.haxx.se/dev/readme-multi_socket.html).
That's an incorrect interpretation of what I've written on that page.
libevent needs time it says, but libevent is "just" a wrapper around the
underlying mechanism which in my case was epoll() on Linux. Of course I urge
you to run your own tests and measurements and not just trust my results, but
I would bet that using a clever event library will not make any noticable
difference in transfer speeds compared to using epoll directly.
I wanted to use epoll to get maximum performance (read the link:
http://devblog.seomoz.org/2011/02/high-performance-libcurl-tips/ and so
eager to see that type of numbers in my application as well). Any help (like
providing some sample or some light) towards achieving this is greatly
appreciated.
Those tips are basically just describing how the multi_socket API works.
If you insist on using epoll instead of an event library, just walk through
the example code and replace the event library calls to calls to epoll
directly!
--
/ daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
