Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Christian Kohlschuetter
Am Samstag, 11. Oktober 2003 17:56 schrieb Michael Becke: > I agree with Oleg here. The real problem is the inability to terminate > a request and force close a connection. Having this functionality would > help in this case as well as many others. > > Mike I also agree that we need such functi

Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Michael Becke
I agree with Oleg here. The real problem is the inability to terminate a request and force close a connection. Having this functionality would help in this case as well as many others. Mike Oleg Kalnichevski wrote: I think the shortest example to demonstrate HTTPClient looping endlessly is

Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Oleg Kalnichevski
> I think the shortest example to demonstrate HTTPClient looping endlessly is > pointing to a chargen-Service: > > Enable chargen in inetd an try to access http://localhost:19/ > > It really does not matter from which side (server/client/user) the problem > arises, but it is HttpClient's job to

Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Christian Kohlschuetter
Here is the new patch (attached to this mail). Index: org/apache/commons/httpclient/AutoCloseInputStream.java === RCS file: /home/cvspublic/jakarta-commons/httpclient/src/java/org/apache/commons/httpclient/AutoCloseInputStream.java,v r

Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Christian Kohlschuetter
Am Freitag, 10. Oktober 2003 18:02 schrieb Kalnichevski, Oleg: > > I can easily provide test cases which will cause HttpClient to eat up all > > available memory throwing an OutOfMemoryError because of reading and > > reading from a never ending HTTP Response. > > > > I would regard this behaviour

Re: Proposal: Configurable HTTP Response length limit

2003-10-11 Thread Christian Kohlschuetter
Am Freitag, 10. Oktober 2003 18:56 schrieb Eric Johnson: > (...) If we took > your suggestion and simply stopped reading the stream sent by the > server, when processing the _next_ request on the same socket, we'd run > into lots of problems. Eric, I think this is the problem of my workaround, wh

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Christian Kohlschuetter
I completely agree with you that we all should write standards-compliant HTTP web pages/CGI programs/Servlets etc. Unfortunately, it is not always in our hands. As nearly everybody can write PHP scripts today, clients which attempt to read from them, should be error-tolerant. If you want to ru

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Eric Johnson
Christian, I would add to my previous two suggestions about using HTTP 1.0 level support and the "Connection:close" header, an additional suggestion. Provide your own connection manager that does not attempt to re-use connections in a persistent manner, or at least, lets you control whether t

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Christian Kohlschuetter
Am Freitag, 10. Oktober 2003 23:40 schrieb Oleg Kalnichevski: > Chris, > Please see my comments in-line I think it is less the patch itself but a standpoint problem (standards-compliant vs. robust). To end this exasperating thread: everybody is invited (but not forced) to use the provided patch

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Oleg Kalnichevski
Chris, Please see my comments in-line On Fri, 2003-10-10 at 18:29, Christian Kohlschuetter wrote: > I completely agree with you that we all should write standards-compliant HTTP > web pages/CGI programs/Servlets etc. > > Unfortunately, it is not always in our hands. As nearly everybody can write

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Christian Kohlschuetter
I completely agree with you that we all should write standards-compliant HTTP web pages/CGI programs/Servlets etc. Unfortunately, it is not always in our hands. As nearly everybody can write PHP scripts today, clients which attempt to read from them, should be error-tolerant. If you want to ru

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Eric Johnson
If you are determined to work with a server that sends limitless responses (an odd use of HTTP - I'd recommend an alternative protocol such as BEEP, with its chunks that you could consume forever. Keep in mind, for example, that there could be a proxy server in the middle that thinks it should

RE: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Kalnichevski, Oleg
> I thought HttpClient was a client for the _real-world_ HTTP-Servers, just as > the HTTP Clients of modern web browsers are/should be (I guess that's why > there is a switch to enable/disable "strict" mode in HttpClient). If those so called real world servers do silly stuff, the fix should be a

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Christian Kohlschuetter
Am Freitag, 10. Oktober 2003 18:02 schrieb Kalnichevski, Oleg: > > I can easily provide test cases which will cause HttpClient to eat up all > > available memory throwing an OutOfMemoryError because of reading and > > reading from a never ending HTTP Response. > > > > I would regard this behaviour

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Eric Johnson
At a lower level, the potential failure points are undoubtedly there. Unless you could point to a real-world server that causes them, or a security bug that stems from them, any such fixes would be academic in nature. Can you cause an out of memory failure while using getResponseBodyAsStream(

RE: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Kalnichevski, Oleg
> I can easily provide test cases which will cause HttpClient to eat up all > available memory throwing an OutOfMemoryError because of reading and reading > from a never ending HTTP Response. > > I would regard this behaviour as a bug. Sure. The bug in the software on the server side. And the f

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread ck
Am Freitag, 10. Oktober 2003 17:01 schrieb Eric Johnson: > I would think that if your application has any reason to believe that > the response will be unbounded, then you should use > getResponseBodyAsStream. > > I suppose would could add functions that took a limit parameter for the > functions g

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Eric Johnson
I would think that if your application has any reason to believe that the response will be unbounded, then you should use getResponseBodyAsStream. I suppose would could add functions that took a limit parameter for the functions getResponseBody() and getResponseBodyAsString(). Something like:

RE: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Kalnichevski, Oleg
MAIL PROTECTED] Sent: Friday, October 10, 2003 16:12 To: Commons HttpClient Project Subject: Re: Proposal: Configurable HTTP Response length limit Am Freitag, 10. Oktober 2003 13:33 schrieb Ortwin Glück: > Chris, > > Thanks for posting. However I really don't see why this sho

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread ck
Am Freitag, 10. Oktober 2003 13:33 schrieb Ortwin Glück: > Chris, > > Thanks for posting. However I really don't see why this should be a > responsibility of HttpClient. The user can always discard the rest of a > response if he wants to. I my eyes you are solving the problem at too > low a level.

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Michael Becke
Hi Christian, I second/third Odi's and Oleg's sentiments. This functionality really belongs in a layer above HttpClient. If you do create something that fits into this niche it would be welcomed in the HttpClient contribs directory. Mike On Friday, October 10, 2003, at 06:47 AM, Christia

RE: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Kalnichevski, Oleg
Glück [mailto:[EMAIL PROTECTED] Sent: Friday, October 10, 2003 13:34 To: Commons HttpClient Project Subject: Re: Proposal: Configurable HTTP Response length limit Chris, Thanks for posting. However I really don't see why this should be a responsibility of HttpClient. The user can always di

Re: Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Ortwin Glück
Chris, Thanks for posting. However I really don't see why this should be a responsibility of HttpClient. The user can always discard the rest of a response if he wants to. I my eyes you are solving the problem at too low a level. The only problem that arises is with endless streams, since Http

Proposal: Configurable HTTP Response length limit

2003-10-10 Thread Christian Kohlschütter
Hello, since this is my first posting to this list, first of all, let me thank all of you for writing/helping to improve jakarta-commons httpclient. I would like to present a tiny patch which enables the user to set a maximum number of bytes to read from a HTTP response. This patch has proven