Quintin Beukes wrote:
Hey,
I made some basic servlets to test throughputs for HttpClient,
HttpCore and sun's sun.net.www.protocol.http.HttpURLConnection (or
something along those lines, I don't have an API at hand now).
I basically just put up an apache session listening on port 8580, and
then Tomcat on port 80. In tomcat I dumped 3 servlets, each using one
of the 3 methods to retrieve a 51kb page from the apache session.
The benchmarks was done with "ab" with 10 second timeouts. The results
are as follows (in byte readings, not bit readings).
Doing a direct request to apache I get a throughput of about 33mb/s.
Using HttpURLConnection, 4.7MB/s.
Using HttpCore 3.6MB/s
Using HttpClient 6.03mb/s
The request times were in general best with apache/HttpClient, maxing
at 38ms for the longest request.
This was done at a concurrency of 1 connection. Pushing this up to 5,
the HttpCore's connection times stayed the same, but throughput
dropped to 980kb/s. HttpClient's connection times now came to the same
as HttpCore (230ms for longest request), but it's throughput increased
to 7.1mb/s.
This is all good and well, but my question is this. Are there any ways
I can increase these throughputs? Before just jumping into
optimizations suggestions and referals to documentation, could someone
perhaps just enlighten me into why these speeds are so different. And
especially so low?
I do not want to jump into conclusions here, but I seriously suspect
something is wrong with your test cases. In particular I do not
understand why you are using Tomcat to host the code your are testing.
What is the point of that?
I made an extensive comparison of HttpCore vs HttpClient (both versions
3.1 and 4.0). HttpCore can easily achieve a throughput of 10,000 (small)
requests per second on a fairly average hardware.
http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore
I am obviously using HttpCore wrong for it to be so slow (I mean,
HttpClient is built on core, so it's impossible to be faster). It's
just that the documentation for these are sparse, and I'm having a
very hard time finding my way around.
And beyond this, Tomcat itself gives me fantastic speeds reading
static data. Where are the bottlenecks?
The HttpCore setup I used was the "Basic GET Requests" example
available on the site. I modified it a bit to allow URL input and
target server specified through parameters passed to the servlet.
Beyond that I didn't change anything. My assumption is that
connections aren't being reused. In fact, I know this is the case, so
it logically and obviously removes the validity of the above
benchmarks for HttpCore. I'm not going to remove them though.
I would really like to get throughputs similar to direct Apache
connections (at least 33+ mbyte/s, which is round about the general
limit of hard drive transfer rates). Full ethernet throughput is
preferable and I'm sure possible. Downloading large files I get speeds
of up to 54mbyte/s with Apache. I'm sure this is limited only to the
hard drive I tested with, as it's a gigabit connection, and this is
just less than half it's full potential. I haven't done tests using
memory transfer. Will try it when I can diagnose the bottlenecks I
currently have, and then use them as measurements to gain insights
into the speed of the program itself by taking as much of the other
bottlenecks in the system away as possible (in this case the
limitation of data transfer rates from the hard drives).
The three benchmark servlets I used are here:
HttpClient: http://quintin.dev.junkmail.co.za/httpclient/Bench.java
HttpCore: http://quintin.dev.junkmail.co.za/httpclient/BenchCore.java
HttpURLConn: http://quintin.dev.junkmail.co.za/httpclient/BenchURL.java
And a utility class I use in the above:
http://quintin.dev.junkmail.co.za/httpclient/StreamBinder.java
Any insights would be greatly appreciated.
I'll try to look at the benchmark code over the weekend. But I would
_strongly_ suggest measuring the performance of each respective
framework by using direct connections to the target server. Once you
have established the baseline you can work on optimizing the code when
running inside a servlet container.
Oleg
regards,
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
