Re: strange behavior of 2.0-rc1

Yes, it works. THX Leo Michael Becke wrote: Okay, I think I have the fix. It seems that there is a problem releasing connections that throw an exception then closing the response stream. Leo, please try out the patch attached to . M

Re: strange behavior of 2.0-rc1

Okay, I think I have the fix. It seems that there is a problem releasing connections that throw an exception then closing the response stream. Leo, please try out the patch attached to . Mike On Saturday, August 30, 2003, at 12:59 P

Re: strange behavior of 2.0-rc1

Well, I've narrowed it down a little. It seems that connections made to the host below are not being released. Coincidentally all of these hosts use chunked encoding. I don't know if this is the cause, but it seems suspicious. I'm going to run some more tests. Attached is the perl script I'

Re: strange behavior of 2.0-rc1

Hi Leo, Well, I've done some analysis of the log file and I think I have a few answers. It seems that a few connections are not being released. To be exact, 47 connections are not released. 42 of them get reclaimed by the garbage collector, but the other 5 remain in limbo. We'll need to

Re: strange behavior of 2.0-rc1

Hi Mike, you have it here: http://com-os2.ms.mff.cuni.cz/temp/example.log.gz (2767808B) I set maxTotalConnection=5. -g- Michael Becke wrote: Hi Leo, Attached at the bottom of my message is another log4j configuration. This should considerably decrease the amount of logging. It also keeps

Re: strange behavior of 2.0-rc1

Hi Leo, Attached at the bottom of my message is another log4j configuration. This should considerably decrease the amount of logging. It also keeps all of the logging info in a single file. To successfully debug this case we will need all of the logging data for a particular session. In

Re: strange behavior of 2.0-rc1

Hi Mike, I've never used log4j and I missed your logging.html page, thus I tried my simple config (see below). Unfortunately, I cannot reproduce the bug easily (-> no test case yet) - the robot has to run more than 6 hours. BTW: I had to rotate log files due to a huge volume of data, so you can

Re: strange behavior of 2.0-rc1

Hi Leo, Could you give me a little more detail about what these various files are? Do each of them show the problem that's occurring? Also, though it's a little late now, we really only need the debug log for MultiThreadedHttpConnectionManager and perhaps the wire I think. Have you had

Re: strange behavior of 2.0-rc1

Mike, the the logs are here: http://com-os2.ms.mff.cuni.cz/temp/ (example.*.*) THX -g- Michael Becke wrote: Yea, "good news" :) So now we know that we're losing connections somewhere, and they are not being garbage collected. There are a bunch of things we can try.Here are a few ide

Re: strange behavior of 2.0-rc1

As long as the connection is released, as it is via method.releaseConnection(), all is okay. There are three ways to force a connection close: 1) Create an HttpMethod that either overrides HttpMethodBase.shouldCloseConnection() to always return true or that calls setConnectionCloseForced(tr

Re: strange behavior of 2.0-rc1

OK, before I start with all the points, can you tell me whether the following strategy may cause the problem? In Network.java you can see, that the response shorter than 64kB is read completely, but when it is longer, just first 64kB are read. In both cases, method.releaseConnection is called.

Re: strange behavior of 2.0-rc1

Yea, "good news" :) So now we know that we're losing connections somewhere, and they are not being garbage collected. There are a bunch of things we can try. Here are a few ideas: - enable debug logging for the MultiThreadedHttpConnectionManager. You will probably also want to print out

Re: strange behavior of 2.0-rc1

Hi Mike, "good news": c-0.log:org.apache.commons.httpclient.HttpException: Timeout waiting for connection c-0.log-at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:314) c-0.log-at org.apache.commons.httpc

Re: strange behavior of 2.0-rc1

Hello Leo, On Sunday, August 24, 2003, at 04:04 PM, Leo Galambos wrote Hi Mike, what do you suggest? Your tutorial does not cover this - a) you use System.exit(-1) which would not be a good solution in my case ;-); or b) you do nothing, which does not work, because "executeMethod" would throw s

Re: strange behavior of 2.0-rc1

Michael Becke wrote: Also, in you example Network.java class I noticed the following lines: while (r.status == -1 && attempt++ < RETRIES) { try { r.status = client.executeMethod(method); r.msg = null; } catch (Th

Re: strange behavior of 2.0-rc1

Hi Leo, These 20 CLOSE_WAIT sockets are from the 20 connections in the pool held by MultiThreadedHttpConnectionManager. When connections are released they are not necessarily closed. My guess is that connections are being lost somewhere. If all 20 connections are checked out and not released

Re: strange behavior of 2.0-rc1

Laura Werner wrote: Leo Galambos wrote: I am using httpclient (HC) in a webcrawler. After 6 hours of run, HC stops working and I think, it is locked by some lock of a critical section in HC. The "problematic" code, I use, is here: http://www.egothor.org/temp/Network.java Are you running on W

Re: strange behavior of 2.0-rc1

Leo Galambos wrote: I am using httpclient (HC) in a webcrawler. After 6 hours of run, HC stops working and I think, it is locked by some lock of a critical section in HC. The "problematic" code, I use, is here: http://www.egothor.org/temp/Network.java Are you running on Windows by any chance?

strange behavior of 2.0-rc1

Hi. I am using httpclient (HC) in a webcrawler. After 6 hours of run, HC stops working and I think, it is locked by some lock of a critical section in HC. The "problematic" code, I use, is here: http://www.egothor.org/temp/Network.java When 20 threads are used, I have this in `netstat -an` aft