DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-11 07:26 --- Adrian, this is a wise (+1-worthy) solution. Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line [EMAIL PROTECTED] changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution||FIXED --- Additional Comments From [EMAIL PROTECTED] 2003-12-11 22:56 --- Patch committed. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 13:08 --- Oleg, in fact, this checked exception is caught and simply logged with LOG.error (see patched HttpMethodBase.responseBodyConsumed()). A simple solution would to be to remove this LOG.error(...) statement (just as in closeSocketAndStreams) - at least if WARN_EXTRA_INPUT is false If you really wish that this potential IOException will not be triggered at all, I suggest an option to have the already discussed TEST_EXTRA_INPUT modes disabled/enabled and silent/enabled with logging. Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 13:17 --- in fact, this checked exception is caught and simply logged with LOG.error (see patched HttpMethodBase.responseBodyConsumed()). Christian, Unfortunately I can't agree that this is just a matter of ignoring the said checked exception. The problem is that there's no (or I personally do not know of a) reliable way to tell if the I/O exception is thrown due to a closed socket or due to a communication error. The former can be safely ignored but the latter cannot. Unless you know how to resolve this problem without resorting error message parsing, I persnally would rather see this check removed altogether. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND
INSERTED IN THE BUG DATABASE.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560
HttpClient loops endlessly while trying to retrieve status line
--- Additional Comments From [EMAIL PROTECTED] 2003-12-10 13:39 ---
Oleg,
it really does not matter what caused that IOException (no parsing needed).
If an IOException is thrown, the connection is not reusable.
Excerpt from patched responseBodyConsumed():
try {
if(responseConnection.isResponseAvailable()) {
(...)
setConnectionCloseForced(true);
}
} catch (IOException e) {
LOG.error(Unexpected I/O error when testing for extra response data, e);
setConnectionCloseForced(true);
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 13:58 --- Maybe I am getting paranoid, but it still matters to me. I do not like the idea of just ignoring exceptions altogether, or putting a error/warning message for what is essentially a normal behaviour. Folks, do you think it is OK if we just ignored I/O exceptions in this situation? Please let us know Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 14:44 --- Oleg, IMHO, if a specific thrown Exception is regarded as part of some normal/tolerable behaviour, it is safe to ignore that Exception (even without logging). This seems to be the case here! Our potential IOException is basically thrown by InputStream.available(), which means that there was a problem with retrieving the number of readable bytes from the stream. The exact reason for this problem does not matter. The IOException is a reliable signal for HttpClient that the HTTP connection should not be reused. By the way, the following two methods also silently ignore Exceptions. Do you think that this behaviour needs to be changed? HttpConnection.shutdownOutput() HttpConnection.closeSocketAndStreams() Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 15:04 --- By the way, the following two methods also silently ignore Exceptions. Do you think that this behaviour needs to be changed? In both cases, I believe it is OK to ignore I/O exceptions. I would not mind seeing a warning in case of closeSocketAndStreams, though, but I can also live without one. Anyways, feel free to provide a patch. I do not mind checking in the patch with I/O exceptions silently ignored, if +1 from at least one committer is secured. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-10 22:25 --- I do not mind checking in the patch with I/O exceptions silently ignored, if +1 from at least one committer is secured. Oleg, I would be -0 to silentlly ignoring exceptions. That would be a -1 if I'd actually had time to follow HttpClient and do something about it myself. I would suggest that if you expect the exception to happen from time to time and can deal with it by just carrying on that at minimum you log the exception at debug level. That way when people start seeing a problem with HttpClient, we'll tell them to turn on debugging and that exception will become visible and they can then check that it's not the cause of the problem. Adrian. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-09 17:46 --- Christian, Well, I have some bad news. I was working on merging TestBadContentLength patch with the main one (Patch (take 6)) in order to get it prepared to be checked in into CVS trunk. When I reran the tests, I unfortunately discovered that several I/O exception were thrown that were not there before. Aparently those exceptions have been caused by the newer Tomcat (version 4.1.29) which appears to be dropping connection immediately after having sent the response body if the 'connection: close' request header is present in the request. So what is happening is that HttpConnection.isResponseAvailable results in an I/O exception, as the socket is closed on the server side the same moment the last response byte was sent: [ERROR] HttpMethodBase - -Unexpected I/O error when testing for extra response data java.io.IOException: Stream closedjava.io.IOException: Stream closed at java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java, Compiled Code) at java.io.PushbackInputStream.available(PushbackInputStream.java:238) at org.apache.commons.httpclient.HttpConnection.isResponseAvailable(HttpConnection.java:863) at org.apache.commons.httpclient.HttpMethodBase.responseBodyConsumed(HttpMethodBase.java:2263) at org.apache.commons.httpclient.HttpMethodBase$1.responseConsumed(HttpMethodBase.java:1765) at org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoCloseInputStream.java, Compiled Code) at org.apache.commons.httpclient.AutoCloseInputStream.checkClose(AutoCloseInputStream.java, Compiled Code) at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java, Compiled Code) at java.io.FilterInputStream.read(FilterInputStream.java:90) at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:161) at org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java, Compiled Code) at org.apache.commons.httpclient.HttpMethodBase.getResponseBodyAsString(HttpMethodBase.java:740) at org.apache.commons.httpclient.TestMethodsLocalHost.testMethodsGet(TestMethodsLocalHost.java:174) This is of course a fringe case, but the whole idea of having HttpConnection.isResponseAvailable was to make HttpClient more reliable, and not less reliable. We clearly overlooked this scenario. In Java prior to version 1.4 there is no way to test if the socket has been closed on the other side, so using Socket#isClosed is not an option. I am afraid HttpMethodParams.WARN_EXTRA_INPUT stuff must be completely removed. The offending code needs to be removed, and the whole process will have to start from the very start. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-08 15:24 --- Created an attachment (id=9441) Patch for CVS HEAD's TestBadContentLength - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-08 15:28 --- Odi, please check this patch (matching HttpClient's old behaviour without the ProtocolException suggestions). Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-12-05 21:26 --- Chris, the TestBadContentLength test is now in CVS HEAD but it still fails and is not included in the test nohost suite. Could you take care of it please? Cheers. Odi - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-27 10:23 --- I would just like to ask if there are any objects against the patch, or if Oleg can commit it. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-20 22:05 --- Created an attachment (id=9221) Patch (take 6) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-20 22:11 --- It turned out that Christian's patch broke 3 test cases. In fact there's nothing wrong with the patch itself. It is just SimpleHttpConnection stuff is too flacky: its isResponseAvailable method does not seem to be able to produce correct results. I worked the problem around in all three cases. It is time we replaced SimpleHttpConnection with something a bit more robust I am +1 to commit Christian's patch Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 12:09 --- Created an attachment (id=9189) Revised patch; please review - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 12:20 --- Please have a look at the revised patch (also see mailing list discussion). I have changed Oleg's suggested TEST_EXTRA_INPUT to LOG_EXTRA_INPUT (we always test but log only in strict mode), and reversed the -1 to Integer.MAX_VALUE (terminates as opposed to the -1 solution, and saves one comparison per loop cycle). Moreover, there is no difference between empty and garbage lines anymore. Both of them are taken into account for LOG_EXTRA_INPUT, which makes no difference for lenient mode, anyway. The suggested CRLF-only checking is simply omitted, as it would make things just too complicated without any advantage (the connection is closed anyway in case of a wrong Content-Length header etc.). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 13:04 --- Christian, 1. Please DO NOT use System.out.println()! This is a big NO-NO in HttpClient. Please use commons-logging capabilities instead 2. Do you mind renaming LOG_EXTRA_INPUT to WARN_EXTRA_INPUT? 3. Integer value –1 used extensively throughout HttpClient to denote that an integer parameter is disabled (not applicable). I would prefer this convention to apply to STATUS_LINE_GARBAGE_LIMIT for the consistency sake, even at the expense of a few CPU cycles. Otherwise looks OK to me. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 13:21 --- Gosh! Shame on me. This System.out.println() was left over from debugging... Please remove it while I am standing in the corner. The problem with -1 vs. Integer.MAX_VALUE is that checking is not disabled in this case. It will just terminate after reading Integer.MAX_VALUE (2^31-1) lines, whereas -1 won't. Feel free to apply modifications, I think I can live with them :) Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 13:43 --- There's no need to stand in the corner. I believe none of us is a five-year-old here. Apart from left-over println() statement my comments were merely hair picking. I leave it to your discretion to decide what changes need to be made to the patch. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-19 13:56 --- Created an attachment (id=9191) Final (?) patch - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
On Monday 17 November 2003 21:05, Oleg Kalnichevski wrote: I have not found any mentioning of unexpected content in the RFC, so this is another reason why I would be a bit cautious about throwing a protocol exception. It would suffice to spit out a warning, drop the connection and move on. Alright. I can live with that. :-) Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Christian, Feel free to make changes to the patch that you deem necessary. Once you think it is ready, I suggest we once again ask all the interested parties to raise their objections and express concerns. If there's no significant opposition to the final revision of the patch, and it is OKed by at least by two committers, I'll consider it good to be checked in. Oleg -Original Message- From: Christian Kohlschütter [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 18, 2003 11:13 To: Commons HttpClient Project Subject: Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line On Monday 17 November 2003 21:05, Oleg Kalnichevski wrote: I have not found any mentioning of unexpected content in the RFC, so this is another reason why I would be a bit cautious about throwing a protocol exception. It would suffice to spit out a warning, drop the connection and move on. Alright. I can live with that. :-) Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
[EMAIL PROTECTED] wrote: --- Additional Comments From [EMAIL PROTECTED] 2003-11-17 14:38 --- Odi, agreed - whitespace is a wrong term. CRLF is better. CRLF or LF is correct ;-) (see RFC2616, section 19.3). Would you then prefer my first version of the patch, or do you have another idea how to handle this? Sorry I did not look at the patch. I just outlined my idea of how it should be in my opinion. Odi Ps. Please use the list for discussion and only post decisions to Bugzilla. Short lists of bug notes are read quicker when fixing bugs. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Odi, That would be REALLY cool! A simple authenticating proxy (or a proxy that could effectively 'fake' popular authentication schemes) would be a very much appreciated contribution. By the way, have a look at the Christian Kohlschütter's SimpleHttpServer: http://nagoya.apache.org/bugzilla/showattachment.cgi?attach_id=9066 I think that can be a good starting point for a better framework than SimpleHttpconnection. Oleg On Mon, 2003-11-17 at 15:00, [EMAIL PROTECTED] wrote: DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-17 14:00 --- Oleg, thank you for reviewing my patch. I think the reviewed version is OK in general (AFAICS from the diff - I haven't applied it yet). Just a few comments (new ideas?) by me: In my opinion, strict mode should be very pedantic about standards compliance. HttpClient should notify the user wherever a problematic, non-standards situation is detected. Of course, trailing whitespace should be silently ignored, but any other characters should be regarded as unexpected (is there a section in RFC 2616 for that? I haven't found it yet). The question is: Is (non-whitespace) garbage following the response (caused by a wrong Content-Length header, for example) unexpected enough? In your version of the patch, there is no chance to get informed about such a situation - and in 'lenient' mode, the detection is disabled completely (did you check the TestBadContentLength testcase? does it pass?). Regarding the ProtocolException/ResponseConsumedWatcher thing, of course, it _is_ a workaround to get that Exception thrown to the caller. However, I would appreciate it if the user _would_ receive that Exception (somehow). I even think it is not such a bad idea to keep that in responseConsumed(), just to inform every HttpClient component that there was an error while reading the response (the interface is not public, anyway). Instead of throwing an Exception, we could also have a boolean without/with errors return value, of course... In short, I would prefer the following behaviour: - For any mode: If garbage is detected, read (up to a certain limit of bytes N) until end of garbage (maximum of N bytes) or until a non-whitespace character is received; N is something 10 (should be user-definable). - For any mode, close the connection (the conncetion is definitely unreliable). - For strict mode, throw a ProtocolException if anything else but whitespace has been received. - (Optionally) introduce an extra pedantic mode (inherits strict mode) and throw a ProtocolException even if N bytes of _whitespace_ garbage have been received. Best regards, Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Christian, see my comments in-line In my opinion, strict mode should be very pedantic about standards compliance. HttpClient should notify the user wherever a problematic, non-standards situation is detected. I do not mind being over-pedantic, but not at the expense of code quality. My personal impression was that throwing a protocol exception from responseConsumed required too much of an ugly plumbing. I believe that a big fat log warning should be enough. A protocol exception thrown from responseConsumed would not make HttpClient more reliable (dirty connection would be dropped, anyway), it would just make it, well, more pedantic. That is it. Of course, trailing whitespace should be silently ignored, but any other characters should be regarded as unexpected (is there a section in RFC 2616 for that? I haven't found it yet). This is what RFC has to say: quote In the interest of robustness, servers SHOULD ignore any empty line(s) received where a Request-Line is expected. In other words, if the server is reading the protocol stream at the beginning of a message and receives a CRLF first, it should ignore the CRLF. /quote http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.1 I have not found any mentioning of unexpected content in the RFC, so this is another reason why I would be a bit cautious about throwing a protocol exception. It would suffice to spit out a warning, drop the connection and move on. Folks, any strong options on this issue - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
I have not found any mentioning of unexpected content in the RFC, so this is another reason why I would be a bit cautious about throwing a protocol exception. It would suffice to spit out a warning, drop the connection and move on. Folks, any strong options on this issue I would prefer a warning to an exception. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-13 13:27 --- Created an attachment (id=9093) Simple, generic HTTP Server; used TestBadContentLength as example - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-13 13:40 --- Oleg, thank you for compliments :) I also think that it is quite important for a HTTP client to have a bundled, tiny HTTP server for testing. Unfortunately, at the moment, I do not have the time to dig that deep into HttpClient's test cases. However, I provide a simple HTTP server framework, where you can construct the replies yourselves (anyone interested in?) By default, the SimpleHttpServer does nothing but throwing a 503 Service Unavailable until you extend it with your own request handlers. One is provided in the new TestBadContentLength. Another one is a handler chain, where multiple handlers can be stacked up, even to very complex structures (chains of chains). The chain stops on the first handler that writes to the given output stream. By the way: It is not very well tested, just give it a try. Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line [EMAIL PROTECTED] changed: What|Removed |Added Severity|Normal |Enhancement Status|NEW |ASSIGNED Priority|Other |Low Target Milestone|--- |2.1 Beta 1 --- Additional Comments From [EMAIL PROTECTED] 2003-11-13 19:05 --- Christian, I finally found enough time to thoroughly examine your patch. While OK in general there were a few minor details that I did not quite like: - Throwing a protocol exception from ResponseConsumedWatcher#responseConsumed was not such a good idea after all. I think it requires too much of ugly code for very little (or none at all) practical gains. In my version of the patch extra response content results in a warning message. No exception is thrown. - HTTP spec requires that HTTP agents ignore all blank lines that precede the response status line. In your patch is was not taken into consideration - I adjusted the new parameters to be more in line with existing parameter conventions in HttpClient Bottom line, I did go a bit rough on your patch. Please let me know if these changes are OK with you or not. If there’s something you do not quite agree with, let me know I'll take a closer at your simple HTTP Server stuff a bit later. I am quite determined to replace the existing SimpleHttpConnection testing framework with a new one based on your code. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-13 19:09 --- Created an attachment (id=9095) Patch (take 2) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-13 22:34 --- Created an attachment (id=9096) Patch (take 3) Contains a minor bug fix - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-12 10:23 --- Created an attachment (id=9064) New patch, includes surplus response data check on first request (see mailing list) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-12 10:24 --- Created an attachment (id=9065) new version of TestBadPorts - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-12 10:24 --- Created an attachment (id=9066) New test: TestBadContentLength - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-12 17:03 --- Chritian, I am a bit stressed out right now, but I'll try to take a closer look at your patch tonight. There are things I would possible like to see done differently. Feedback to follow. But I REALLY like your TestBadContentLength testcase. We all would greatly appreciate if you could look at the possibility of replacing the current SimpleHttpConnection based testing framework with something of based on your approach. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 07:47 --- Oleg, I don't think that it would be a good idea to make that a configurable parameter. RFC2616 states that the very first line of the response _must_ be the status line (No CR or LF is allowed before that). But I guess that you have made bad experiences with some servers, that is why you have implemented the loop readStatusLine at all, haven't you? If so, then maxLinesToInspect should be set to a reasonable, fixed value (any value between 2 and 5 should be ok), because if the loop _is_ looping, you are already breaking the standard (a strict-mode check would make sense here!) If not, we can remove the loop completely (and can forget that configurable parameter :-) Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 08:33 --- Sooner or later somebody will request this as a configurable parameter. We have a very wide range of users. So we will need every parameter to be configurable. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 09:09 --- Christian, The problem here is persistent HTTP/1.1 connections. Occasionally some broken SGI scripts report content-length value that in fact does not reflect the real length of the content body sent across the wire. Consider the following: --- HTTP/1.1 200 OK Content-Length: 5 Screw you Really bad HTTP/1.1 200 OK Content-Length: 9 Ooopsie --- With the status line read loop in place HttpClient at least stands a chance of graceful recovery from this kind of mishap. Unfortunately in this situation I do not see an alternative to just scanning input stream for something that may (or may not) happen to be a valid response line So, it is paramount that the suggested check is off per default. Only in very special cases like yours one may choose to activate it. That's why the parameter is a must Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 09:54 --- Created an attachment (id=9052) Modified patch, maximum garbage lines default to Integer.MAX_VALUE - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 09:55 --- Created an attachment (id=9053) Improved testcase - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 09:58 --- Oleg, good argument. I have attached the new patch (as well as an improved testcase). Now, in lenient mode, the maximum number of garbage lines is Integer.MAX_VALUE (following your argumentation, 5 is too little). In strict-mode, this number is set to 0 by default. Is this OK? Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 10:39 --- Oleg, another idea: In case of reusing persistent connections, wouldn't it be better to simply read/skip any available bytes before sending the next Request header? Because conversation is rather like this: (+ - GET / HTTP/1.1 - Host: www.foo.com - - HTTP/1.1 200 OK - Content-Length: 5 - - * you - Really bad - GET /bar HTTP/1.1 - Host: www.foo.com - - HTTP/1.1 200 OK - Content-Length: 9 Before sending the second GET, just read the surplus you\r\nReally bad off the stream (throw an Exception or skip it/append it to the response, according to strict/lenient mode). I guess that would be a cleaner handling (as the surplus garbage belongs to the first method request, not to the second one). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 10:44 --- Chris, that is very unreliable. It is not guaranteed as to WHEN packets arrive. After you have read all available data, the server may still continue sending more garbage... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
[EMAIL PROTECTED] wrote: Odi has a point here. There is no reliable way of telling where one response body ends and another one starts, if content length information is messed up. That's why the Content-Length header is part of the HTTP 1.1 protocol in the first place. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 11:25 --- Agreed. In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? In this case, at least, I would suggest to close the current connection and open a fresh one. Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
[EMAIL PROTECTED] wrote: Agreed. In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? In this case, at least, I would suggest to close the current connection and open a fresh one. Christian This is the pragmatic way of solving the problem. It always works and is very reliable. But it is expensive. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Am Dienstag, 11. November 2003 12:32 schrieb Ortwin Glück: [EMAIL PROTECTED] wrote: Agreed. In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? In this case, at least, I would suggest to close the current connection and open a fresh one. Christian This is the pragmatic way of solving the problem. It always works and is very reliable. But it is expensive. Has this already been implemented or should we do it now?
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Am Dienstag, 11. November 2003 12:32 schrieb Ortwin Glück: [EMAIL PROTECTED] wrote: Agreed. In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? In this case, at least, I would suggest to close the current connection and open a fresh one. Christian This is the pragmatic way of solving the problem. It always works and is very reliable. But it is expensive. btw, what do you mean with expensive? I think an available() check or read()/unread() call pair at the right point would be enough. Christian
RE: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? Christian, Again, there's a similar problem. How long do you think we should wait on a connection to be really sure that there's no garbage coming out of it? I think this timeout is the performance hit Odi was talking about. Besides, regardless how long you wait there can never be 100% certainty that the connection is completely 'clean'. You could still potentially get some garbage out of persistent connection when reading the status line of the following response. So, we are pretty much back to where we started You are welcome to come up with a better solution than the exiting, but I am afraid there's none that it is truly bullet-proof. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Am Dienstag, 11. November 2003 13:01 schrieb Kalnichevski, Oleg: In other words, can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes available than specified with Content-Length? Christian, Again, there's a similar problem. How long do you think we should wait on a connection to be really sure that there's no garbage coming out of it? I think this timeout is the performance hit Odi was talking about. Besides, regardless how long you wait there can never be 100% certainty that the connection is completely 'clean'. You could still potentially get some garbage out of persistent connection when reading the status line of the following response. So, we are pretty much back to where we started You are welcome to come up with a better solution than the exiting, but I am afraid there's none that it is truly bullet-proof. Oleg I perfectly agree - I do not see a bullet-proof solution either. I should correct my assumption: Can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes *INSTANTLY* available than specified with Content-Length Instantly means without waiting/blocking, so at least for this situation, a simple workaround would be feasible. I think that the currently used SocketInputStream's available() method _does_ return values 0. The only additional thing to change is HttpConnection.WrappedInputStream, as it currently lacks an available() method call to the underlying stream. Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Christian Kohlschütter wrote: In this case, at least, I would suggest to close the current connection and open a fresh one. This is the pragmatic way of solving the problem. It always works and is very reliable. But it is expensive. btw, what do you mean with expensive? Expensive: Throwing away the (known bad) connection and opening a new one, is a performance hit of course. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-11 12:50 --- Created an attachment (id=9056) Testcase cleanup: Now uses HttpClient's TimeoutController - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes *INSTANTLY* available than specified with Content-Length Instantly means without waiting/blocking, so at least for this situation, a simple workaround would be feasible. Christian, Fair enough. Feel free to extend your patch to include the check to force-close those connections that report input data available beyond 'content-length' value/terminal chunk. I think that the currently used SocketInputStream's available() method _does_ return values 0. The only additional thing to change is HttpConnection.WrappedInputStream, as it currently lacks an available() method call to the underlying stream. This one is a bug, because it effectively rendered HttpConnection#isResponseAvailable meaningless. I'll provide a fix for it tonight. Thanks for tracking this one down Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
On Tuesday 11 November 2003 13:44, Ortwin Glück wrote: Expensive: Throwing away the (known bad) connection and opening a new one, is a performance hit of course. Sure. But this happens only if we detected a connection to be bad, and then it is a very rare situation. In strict mode, the detection of bad connection should cause an exception to be thrown, instead of reopening another connection. What do you think? Christian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
In strict mode, the detection of bad connection should cause an exception to be thrown, instead of reopening another connection. What do you think? Sounds reasonable. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Eric, Just to clarify things: there's strongno way/strong we touch 2.0 branch for any other reason but fixing a strongserious/strong bug. As far as I am concerned the suggested patch is an enhancement, and not a bug. If any of my comments were interpreted otherwise, I offer my apologies. It was not meant that way. Anyways, if I understand Christian right, the idea is to drop those connections that are known for sure to be screwy, instead of reusing them and getting into a trouble later when processing subsequent responses. The logic in readStatusLine will be enhanced to optionally terminate the infinite loop. That is it. I hope that addresses your concerns somewhat. Oleg -Original Message- From: Eric Johnson [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 15:35 To: Commons HttpClient Project Subject: Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line Christian Kohlschütter wrote: I perfectly agree - I do not see a bullet-proof solution either. I should correct my assumption: Can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes *INSTANTLY* available than specified with Content-Length Instantly means without waiting/blocking, so at least for this situation, a simple workaround would be feasible. I think that the currently used SocketInputStream's available() method _does_ return values 0. Unfortunately, I think that depends. I seem to recall we had difficulties with this function in the past, particularly related to different JVM versions, and also with different implementations of secure sockets. Granted, some of those implementations were/are buggy, but we have to live with them, I think. Before we commit such a change to the 2.0 release branch, we'd have to run it through tests across numerous JVMs on numerous platforms with numerous JCE libraries. We also run the risk that the available function could misbehave not only by giving an incorrect response, but also by blocking for a short period of time (1ms?), which would be disastrous for performance. I think the instantly available criteria is misleading, too. There's absolutely no reason to prevent you from hitting a pathological case where the packet boundary splits right where the extra data is sent, thus leading the instantly available check to return false, even though the data would be read on the subsequent response. In fact, such behavior could be entirely dependent on the misbehaving server. The case that I've encountered stemmed from a server that tossed in an extra CRLF after a previous response. As I recall, the extra CRLF were, not surprisingly, written to the socket as a separate chunk, and therefore stood a good a chance as any of being received by the client as part of a separate packet, and therefore would not be instantly available anyway. In the end, I've got a few concerns: * I'm not sure I see the point of trying to catch the problem at the end of the previous response, which would seem to be the point of the available check, rather than at the beginning of reading the next response. * Are there particular servers that demonstrate bad behavior that we want to catch with this new option? Do we have test cases for those particular servers? * Has it been tested across a myriad of environments? Such changes for the 2.1 CVS HEAD are fine by me. I'm much more concerned about the 2.0 branch, however, and keeping what changes we do there to a minimum. This change seems to straddle the boundary between a bug-fix and additional functionality, at least from where I sit. Then again, I've not looked closely at the patch, I've only followed the discussion. -Eric. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
Oleg, No apologies needed. As far as I'm concerned, your opinions with respect to HttpClient carry far more weight - I seem to contribute a patch about once every three months (and Mike just did a much better rewrite of my latest patch), but otherwise just stir up trouble on the mailing list! One more wrench. It strikes me that if HttpClient does support HTTP pipelining in the future, then an available check will mislead you. Of course there might be data available, as that would be precisely the point. Only when looking at the data can you determine whether or not it is valid to have that data already in the queue Granted, it is perhaps early to worry about pipelining! It seems like we have a spectrum of possibilities to deal with here: Malicious servers -- bad servers with immediate extra data -- bad servers with delayed extra data -- other bad servers (bad cookies et. al.) -- mostly compliant servers -- 100% compliant servers (yeah, right!). It seems like we need to stop the malicious servers from derailing HttpClient. As for the two types of bad servers, I'm not sure I see the point of catching the bad behavior on some requests immediately, when we're guaranteed to catch the problem on a subsequent request 100% of the time. I'm guessing that the minor performance hit of reissuing a request when operating in strict mode will be swamped by the fact that in either case you have to open a new connection? I guess that's part of why I ask whether there are specific badly behaved servers for which enforcing a strict mode with an available check will actually prove useful. It could be that for every bad server out there, they almost all fall into the delayed extra data category, rather than the immediate category. I think my concern comes down to this: How much benefit does HttpClient get out of enforcing the rule in two places, one with the available check, and one with the scan at the beginning of reading the next response? Is the answer 5% of bad servers, or 75% of bad servers? And if you set a strict mode, can HttpClient even operate with those servers anyway? Absent the real-world data, I'd stick with only trying to deal with the bad behavior at the beginning of the next response, rather than at the end of the previous one. -Eric. Kalnichevski, Oleg wrote: Eric, Just to clarify things: there's strongno way/strong we touch 2.0 branch for any other reason but fixing a strongserious/strong bug. As far as I am concerned the suggested patch is an enhancement, and not a bug. If any of my comments were interpreted otherwise, I offer my apologies. It was not meant that way. Anyways, if I understand Christian right, the idea is to drop those connections that are known for sure to be screwy, instead of reusing them and getting into a trouble later when processing subsequent responses. The logic in readStatusLine will be enhanced to optionally terminate the infinite loop. That is it. I hope that addresses your concerns somewhat. Oleg -Original Message- From: Eric Johnson [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 11, 2003 15:35 To: Commons HttpClient Project Subject: Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line Christian Kohlschütter wrote: I perfectly agree - I do not see a bullet-proof solution either. I should correct my assumption: Can we assume that reusing the HTTP connection is unreliable/should be avoided if there are more bytes *INSTANTLY* available than specified with Content-Length Instantly means without waiting/blocking, so at least for this situation, a simple workaround would be feasible. I think that the currently used SocketInputStream's available() method _does_ return values 0. Unfortunately, I think that depends. I seem to recall we had difficulties with this function in the past, particularly related to different JVM versions, and also with different implementations of secure sockets. Granted, some of those implementations were/are buggy, but we have to live with them, I think. Before we commit such a change to the 2.0 release branch, we'd have to run it through tests across numerous JVMs on numerous platforms with numerous JCE libraries. We also run the risk that the available function could misbehave not only by giving an incorrect response, but also by blocking for a short period of time (1ms?), which would be disastrous for performance. I think the instantly available criteria is misleading, too. There's absolutely no reason to prevent you from hitting a pathological case where the packet boundary splits right where the extra data is sent, thus leading the instantly available check to return false, even though the data would be read on the subsequent response. In fact, such behavior could be entirely dependent on the misbehaving server. The case that I've encountered stemmed from a server that tossed in an extra CRLF after
Re: DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
The only additional thing to change is HttpConnection.WrappedInputStream, as it currently lacks an available() method call to the underlying stream. Problem fixed. WrappedInputStream now overrides InputStream#available method. Thanks. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-10 12:50 --- Created an attachment (id=9016) Test connection on localhost's chargen port (and echo), if open - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-10 12:54 --- Created an attachment (id=9018) Suggested patch for HttpMethodBase.readStatusLine - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-10 12:55 --- Please ignore attachment id=9017 (source instead of patch). Look at id=9018 instead. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 24560] - HttpClient loops endlessly while trying to retrieve status line
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24560 HttpClient loops endlessly while trying to retrieve status line --- Additional Comments From [EMAIL PROTECTED] 2003-11-10 14:14 --- Christian, Could you, please, parameterize 'maxLinesToInspect' value using HttpMethodParams class and resubmit the patch? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
