Doug Cutting wrote:
Andy Hedges wrote:
We use Http.BUFFER_SIZE to determine kernel read/write size. So this should be set in params as the sendBufferSize and recieveBufferSize, no?
When you say kernel what are you refering to? I believe but I need to verify that the data is read off the wire as controlled by the obtained inputstream. I might be wrong I need to determine this by sifting though some of the HttpClient code - unless you already know the answer.
I'm talking about the obtained input stream's buffer size, which reads from the socket, as well as the buffer size used by the stream which writes to the socket.
http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpConnection.html#setSendBufferSize(int)
http://cvs.apache.org/viewcvs.cgi/jakarta-commons/httpclient/src/java/org/apache/commons/httpclient/params/HttpConnectionParams.java?rev=1.5&view=auto
The javadoc doesn't show setReceiveBufferSize, but it is in the source.
Doug
Right, now I've looked into it I have discovered a few things. The HttpConnectionParams is new in version 3.0 of HttpClient (the javadoc is for HttpClient 2.0). HttpClient 3.0 is still in alpha. Having said this I have altered the code to work with the new 3.0 code and it passes all the unit tests. I have also implemented the two send/receive buffer configurations.
The new patch is here: http://www.hedges.net/nutch/
3rd time lucky?
-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
Nutch-developers mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
Nutch-developers mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
