Re: Java can leak...
I think the reality here is that the only way to reliably track down the problem is to actually profile the original application that is seeing the problem. There's really not a lot of point in me profiling HttpClient separately because I've not seen any problem with memory leaks so they're not likely to suddenly turn up in profiling. The particular way HttpClient is being used is likely what's bringing out the leak whether the leak is caused by HttpClient code or not. Jan, if you could do some profiling work on your application to identify at least (a) that it is that particular app and not something else on the machine (pretty easy) (b) what types of objects are hanging around and (c) what's still got a reference on them, that would really help track down the problem. The profiling that we've done on our app doesn't show any indication of a problem with HttpClient but the way we use HttpClient is extremely different. Regards, Adrian Sutton. On Friday, May 30, 2003, at 08:49 AM, Wilcox, Mark wrote: Jan, Good point. With Java you don't have to remember as much about silly things like malloc and free in C/C++ which led to all sorts of problems. However, you do have to worry about GC, object creation and holding on to objects for too long. Something like HPJMeter may help in analyzing the problem: http://www.hp.com/products1/unix/java/developers/hpjmeter.html Mark -Original Message- From: Jan Gonsalves [mailto:[EMAIL PROTECTED] Sent: Thu 5/29/2003 6:09 PM To: Commons HttpClient Project Cc: Subject: Java can leak... Oleg, I understand what you mean my mem alloc, but Java can leak, it's a proven fact. Anytime ROOT has a reference to something it makes it unable to GC, however, if you loose your reference and ROOT still has it's, that's a memory leak. ROOT will never GC that memory. Ask youself this, if Memory leaks were impossible in JAVA, how can programs such as JProfiler or OptimizIt sell? Jan Oleg Kalnichevski wrote: Jan, HttpClient is a pure Java application and as such does not manage memory allocation directly. I suggest that you check if there are any known issues with the jvm you are using. Upgrading to HttpClient beta-1 would not hurt, anyways. Oleg On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote: Does anyone know of any memory issues with opening a connection and continuely re-using it, only closing it when a socket timeout happens? I'm currently using ALPHA 3. I have to fake out a browser and navigate through a website to extract information because they don't offer a straight DB connection. I re-use this connection the whole time, never closing it, only recycling it, unless I get a socket timeout, ConnectionException, or HTTPRecoverableException. Our Production server has 4GB of ram. Last night we ran and the system went down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP Client. I've gone over my stuff and it looks relatively solid, I'm wondering now if I'm using HttpClient incorrectly. Does Beta 1 help with any memory issues? Jan Gonsalves - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java can leak...
My impression of your first email was that the server had used up the 4GB of memory. Are you saying it was HttpClient? If so, you must have run the JVM and specified the max heap size to something like -Xmx4GB. If that's the case then this is probably something you don't want to do. Once Java gets over the minimum heap size, specified by -Xms, it will continue to expand all the way up to the max, even if it doesn't need to. Once the JVM allocates memory to the heap it never releases it. Is this what's happening? Mike On Thursday, May 29, 2003, at 06:09 PM, Jan Gonsalves wrote: Oleg, I understand what you mean my mem alloc, but Java can leak, it's a proven fact. Anytime ROOT has a reference to something it makes it unable to GC, however, if you loose your reference and ROOT still has it's, that's a memory leak. ROOT will never GC that memory. Ask youself this, if Memory leaks were impossible in JAVA, how can programs such as JProfiler or OptimizIt sell? Jan Oleg Kalnichevski wrote: Jan, HttpClient is a pure Java application and as such does not manage memory allocation directly. I suggest that you check if there are any known issues with the jvm you are using. Upgrading to HttpClient beta-1 would not hurt, anyways. Oleg On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote: Does anyone know of any memory issues with opening a connection and continuely re-using it, only closing it when a socket timeout happens? I'm currently using ALPHA 3. I have to fake out a browser and navigate through a website to extract information because they don't offer a straight DB connection. I re-use this connection the whole time, never closing it, only recycling it, unless I get a socket timeout, ConnectionException, or HTTPRecoverableException. Our Production server has 4GB of ram. Last night we ran and the system went down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP Client. I've gone over my stuff and it looks relatively solid, I'm wondering now if I'm using HttpClient incorrectly. Does Beta 1 help with any memory issues? Jan Gonsalves - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Java can leak...
We need to do more investigation on our side for sure, I just wanted to know if anyone had noticed memory leaks over long recycled connections. The JVM my app runs in alots 768Mb as the max, but I work in conjunction with serveral apps. I'm the only app using HttpClient. When I run and I really get going processing large chunks of data memory usage for the system really takes a hit. My personal memory footprint based off qps for linux says I max out at about 380mb. If I loose a reference to an object ROOT has, does that count against my memory usage or does that consume memory but no under a particular JVM? My personal belief is that my app does things a new way in our system and I'm putting stress on other objects in the system that aren't normally accustomed to it and they are having difficulty. Anyone know of a good, user friendly (read inexperienced to memory profilling) application? Jan Michael Becke wrote: My impression of your first email was that the server had used up the 4GB of memory. Are you saying it was HttpClient? If so, you must have run the JVM and specified the max heap size to something like -Xmx4GB. If that's the case then this is probably something you don't want to do. Once Java gets over the minimum heap size, specified by -Xms, it will continue to expand all the way up to the max, even if it doesn't need to. Once the JVM allocates memory to the heap it never releases it. Is this what's happening? Mike On Thursday, May 29, 2003, at 06:09 PM, Jan Gonsalves wrote: Oleg, I understand what you mean my mem alloc, but Java can leak, it's a proven fact. Anytime ROOT has a reference to something it makes it unable to GC, however, if you loose your reference and ROOT still has it's, that's a memory leak. ROOT will never GC that memory. Ask youself this, if Memory leaks were impossible in JAVA, how can programs such as JProfiler or OptimizIt sell? Jan Oleg Kalnichevski wrote: Jan, HttpClient is a pure Java application and as such does not manage memory allocation directly. I suggest that you check if there are any known issues with the jvm you are using. Upgrading to HttpClient beta-1 would not hurt, anyways. Oleg On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote: Does anyone know of any memory issues with opening a connection and continuely re-using it, only closing it when a socket timeout happens? I'm currently using ALPHA 3. I have to fake out a browser and navigate through a website to extract information because they don't offer a straight DB connection. I re-use this connection the whole time, never closing it, only recycling it, unless I get a socket timeout, ConnectionException, or HTTPRecoverableException. Our Production server has 4GB of ram. Last night we ran and the system went down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP Client. I've gone over my stuff and it looks relatively solid, I'm wondering now if I'm using HttpClient incorrectly. Does Beta 1 help with any memory issues? Jan Gonsalves - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Java can leak...
Jan, Let me elaborate. No doubt one can still do silly things in Java that may lead to severe memory problems. However, the point I was trying to make is that Java applications do not 'leak' memory in a way non-GC applications do. One can easily starve a java application of heap memory if object allocation de-allocation strategy is flawed. However, no java application should ever consume memory above the limit specified with -Xmx option, unless there is a bug in JVM. I guess you should try narrowing down the problem somewhat. At the moment all we can do is to speculate. If you absolutely positive that the problem is somehow triggered by HttpClient, please let us know some more details on your application's execution environment: OS version, JVM version, application server or servlet engine used, and so on. Oleg -Original Message- From: Jan Gonsalves [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 00:09 To: Commons HttpClient Project Subject: Java can leak... Oleg, I understand what you mean my mem alloc, but Java can leak, it's a proven fact. Anytime ROOT has a reference to something it makes it unable to GC, however, if you loose your reference and ROOT still has it's, that's a memory leak. ROOT will never GC that memory. Ask youself this, if Memory leaks were impossible in JAVA, how can programs such as JProfiler or OptimizIt sell? Jan Oleg Kalnichevski wrote: Jan, HttpClient is a pure Java application and as such does not manage memory allocation directly. I suggest that you check if there are any known issues with the jvm you are using. Upgrading to HttpClient beta-1 would not hurt, anyways. Oleg On Thu, 2003-05-29 at 16:36, Jan Gonsalves wrote: Does anyone know of any memory issues with opening a connection and continuely re-using it, only closing it when a socket timeout happens? I'm currently using ALPHA 3. I have to fake out a browser and navigate through a website to extract information because they don't offer a straight DB connection. I re-use this connection the whole time, never closing it, only recycling it, unless I get a socket timeout, ConnectionException, or HTTPRecoverableException. Our Production server has 4GB of ram. Last night we ran and the system went down to 4MB free, 4! I'm just wondering if I'm leaking due to HTTP Client. I've gone over my stuff and it looks relatively solid, I'm wondering now if I'm using HttpClient incorrectly. Does Beta 1 help with any memory issues? Jan Gonsalves - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
