Re: Session timeout and memory
() - runtime.freeMemory ()).doubleValue(); So this happens to the logs: The memory goes up inside of the session. It sometimes drops a bit but it mainly goes up. So there seems to be a memory leak somewhere... Ok, but then the session is terminated or times out. So I would assume that then the memory is returned as free memory. So the next session should again show used memory lets say 6 MB. But when the previous session had a used memory of lets say 50 MB the next session (created well after the first session is timed out or terminated) will show 50MB + of used memory. And yes: There was only one session present in the testcase and it clearly went up after the first session was timed out without releasing this used memory. Do I have to schedule the instances to get rid of this (as it happens by design) or does it point me to a memory leak in my code which means whatever holds the memory cannot be released even after the session is terminated. Regards Ute Anfang der weitergeleiteten E-Mail: Von: Q [EMAIL PROTECTED] Datum: Di, 12. Sep 2006 01:43:35 Europe/Berlin An: Chuck Hill [EMAIL PROTECTED] Cc: Ute Hoffmann [EMAIL PROTECTED], webobjects- [EMAIL PROTECTED] Betreff: Re: Session timeout and memory On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill% 40global-village.net This email sent to [EMAIL PROTECTED] -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/ practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/blenko% 40martingalesystems.com This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/owen% 40uow.edu.au This email sent to [EMAIL PROTECTED] -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
What you are seeing is almost certainly just the way garbage collection works. When the session times out, the objects it allocated are available for garbage collection. They are not collected at that time, just made available for collection. The garbage collector won't run until it needs to. The latter is not correct. The garbage collector on HotSpot runs frequently and a common occasion is when a response has been returned by a WO application. It appears that it collects on these occasions against the local store. This makes sense in the immediate context, you'd probably have to test it against you own applications to determine whether it makes sense on balance. (I don't believe you can control this behavior in any case). The garbage collector doesn't collect all available memory on these occasions, however. I had a disagreement with someone once about whether the garbage collector would collect a cycle, which it isn't difficult to create, e.g., off of a session. We put up a minimal application on a quiet machine, instrumented the VM, set the session to timeout after 1 minute, requested one page (which created the cycle), and waited. The garbage collector ran many times but it took 10 minutes before the cycle was collected. Behavior of the garbage collector isn't an issue most of the time. If the working set is close to the VM memory size, however, it can cause the VM to thrash and the solution is (usually) to increase the size of VM memory. It can also happen that you know when a sizable chunk of memory is being freed and forcing the gc to run amounts to an optimization, e.g., to reduce paging or total memory use. You can call gc() to advise the garbage collector to run. You can call gc() three times in succession to advise it more strongly to run. Running, in this circumstance, will only cause it to collect what it chooses, it won't necessarily collect all the memory available. Tom Chuck On Sep 11, 2006, at 11:18 PM, Ute Hoffmann wrote: Ok, perhaps I'm looking at the wrong thing. This is the piece of code which I use to determine the used memory: Runtime runtime = Runtime.getRuntime(); double nochVoranden = new Double(runtime.totalMemory () - runtime.freeMemory ()).doubleValue(); So this happens to the logs: The memory goes up inside of the session. It sometimes drops a bit but it mainly goes up. So there seems to be a memory leak somewhere... Ok, but then the session is terminated or times out. So I would assume that then the memory is returned as free memory. So the next session should again show used memory lets say 6 MB. But when the previous session had a used memory of lets say 50 MB the next session (created well after the first session is timed out or terminated) will show 50MB + of used memory. And yes: There was only one session present in the testcase and it clearly went up after the first session was timed out without releasing this used memory. Do I have to schedule the instances to get rid of this (as it happens by design) or does it point me to a memory leak in my code which means whatever holds the memory cannot be released even after the session is terminated. Regards Ute Anfang der weitergeleiteten E-Mail: Von: Q [EMAIL PROTECTED] Datum: Di, 12. Sep 2006 01:43:35 Europe/Berlin An: Chuck Hill [EMAIL PROTECTED] Cc: Ute Hoffmann [EMAIL PROTECTED], webobjects-dev@lists.apple.com Betreff: Re: Session timeout and memory On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin
Re: Session timeout and memory
PROTECTED] Betreff: Re: Session timeout and memory On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill% 40global-village.net This email sent to [EMAIL PROTECTED] -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/ practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/blenko% 40martingalesystems.com This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/owen% 40uow.edu.au This email sent to [EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
3) Can someone point me at some good doc for learning about memory profiling of Java app's. Specifically over a long period of time. Even to figure out how much ram each session is taking and when that is being garbage collected as per Tom's example below. I've used OptimizeIt (http://www.borland.com/us/products/optimizeit/index.html), although not in its most recent incarnation, of which there have been several. I have also heard that JProbe (http://www.quest.com/jprobe) has seen plenty of use with WebObjects, although that might be a few incarnations ago as well. I am aware of JProfiler (http://www.ej-technologies.com/products/jprofiler/overview.html) although I haven't heard of it being used with WebObjects. The first two work by connecting to the VM's JVMPI (Java Virtual Machine Profiler Interface) ports. This slows the VM down a good deal. You wouldn't want to hook up to this in production and leave the profiler running for days until trouble appeared. Some ideas: - Check that sessions are indeed terminating. - I have no idea how much traffic you are seeing or what percentage of overall memory is leaking. However, you might determine how much memory per session is leaking -- that in itself might give you a clue as to the problem, especially if it's a lot. - Set up the app in development with a profiler attached. Set the session timeout to be short. Watch what happens, even with a nominal amount of activity on the session, especially as the session terminates. Also visit any suspicious areas of the application. Be aware that the garbage collector collects in its own sweet time, as I mentioned earlier. The problem is difficult in principal but, especially if the leak is large, it may be pretty easy to spot. I'd also give some consideration to periodically restarting instances. Not as a solution to a real memory problem but as insurance against your mission-critical application failing early Sunday AM. Tom ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
What you are seeing is almost certainly just the way garbage collection works. When the session times out, the objects it allocated are available for garbage collection. They are not collected at that time, just made available for collection. The garbage collector won't run until it needs to. Chuck On Sep 11, 2006, at 11:18 PM, Ute Hoffmann wrote: Ok, perhaps I'm looking at the wrong thing. This is the piece of code which I use to determine the used memory: Runtime runtime = Runtime.getRuntime(); double nochVoranden = new Double(runtime.totalMemory () - runtime.freeMemory ()).doubleValue(); So this happens to the logs: The memory goes up inside of the session. It sometimes drops a bit but it mainly goes up. So there seems to be a memory leak somewhere... Ok, but then the session is terminated or times out. So I would assume that then the memory is returned as free memory. So the next session should again show used memory lets say 6 MB. But when the previous session had a used memory of lets say 50 MB the next session (created well after the first session is timed out or terminated) will show 50MB + of used memory. And yes: There was only one session present in the testcase and it clearly went up after the first session was timed out without releasing this used memory. Do I have to schedule the instances to get rid of this (as it happens by design) or does it point me to a memory leak in my code which means whatever holds the memory cannot be released even after the session is terminated. Regards Ute Anfang der weitergeleiteten E-Mail: Von: Q [EMAIL PROTECTED] Datum: Di, 12. Sep 2006 01:43:35 Europe/Berlin An: Chuck Hill [EMAIL PROTECTED] Cc: Ute Hoffmann [EMAIL PROTECTED], webobjects- [EMAIL PROTECTED] Betreff: Re: Session timeout and memory On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill% 40global-village.net This email sent to [EMAIL PROTECTED] -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
hi ute, Am 12.09.2006 um 08:18 schrieb Ute Hoffmann: But when the previous session had a used memory of lets say 50 MB the next session (created well after the first session is timed out or terminated) will show 50MB + of used memory. 50MB sound a lot for a session. so maybe you’re just warming up the EOF-cache with some big snapshots? any variables stored in the Application? maybe some horrid EOEditingContext? or something in the shared context? atze ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
Ok, perhaps I'm looking at the wrong thing. This is the piece of code which I use to determine the used memory: Runtime runtime = Runtime.getRuntime(); double nochVoranden = new Double(runtime.totalMemory () - runtime.freeMemory ()).doubleValue(); So this happens to the logs: The memory goes up inside of the session. It sometimes drops a bit but it mainly goes up. So there seems to be a memory leak somewhere... Ok, but then the session is terminated or times out. So I would assume that then the memory is returned as free memory. So the next session should again show used memory lets say 6 MB. But when the previous session had a used memory of lets say 50 MB the next session (created well after the first session is timed out or terminated) will show 50MB + of used memory. And yes: There was only one session present in the testcase and it clearly went up after the first session was timed out without releasing this used memory. Do I have to schedule the instances to get rid of this (as it happens by design) or does it point me to a memory leak in my code which means whatever holds the memory cannot be released even after the session is terminated. Regards Ute Anfang der weitergeleiteten E-Mail: Von: Q [EMAIL PROTECTED]> Datum: Di, 12. Sep 2006 01:43:35 Europe/Berlin An: Chuck Hill [EMAIL PROTECTED]> Cc: Ute Hoffmann [EMAIL PROTECTED]>, webobjects-dev@lists.apple.com Betreff: Re: Session timeout and memory On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck On Sep 9, 2006, at 1:51 AM, Ute Hoffmann wrote: Hallo, I would expect that when a session times out that then the used memory in this session is freed imediately... but I see in my memory logging that that seems not to be the case. Any idea what I need to start looking for? Or is this normal behaviour and the memory will be there when it is needed? Regrads, Ute ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill% 40global-village.net This email sent to [EMAIL PROTECTED] -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
On 12/09/2006, at 2:26 AM, Chuck Hill wrote: In addition to garbage collection which has been mentioned, be careful of what you are looking at. If additional memory is allocated to the JVM process, it will not be returned to the operating system until the JVM process terminates. In other words, the total heap size will never decrease but free memory on the heap will vary as objects are created and the garbage collector runs. Chuck It should also be pointed out that this is a good thing not a liability that needs special consideration, because it means that the JVM doesn't need to constantly malloc and free memory. Provided there isn't an actual leak causing the JVM size to continually grow unexpectedly, the virtual memory allocation shouldn't be something to really worry about, as any vm pages that go unused for long enough will eventually get paged out by the OS should the physical memory be required for something else. Assuming your OS has got enough swap allocated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _ / Quinton Dolan - [EMAIL PROTECTED] __ __/ / / __/ / / /__ / _// /Gold Coast, QLD, Australia __/ __/ __/ / / - /Ph: +61 419 729 806 ___ / _\ ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Session timeout and memory
Hallo, I would expect that when a session times out that then the used memory in this session is freed imediately... but I see in my memory logging that that seems not to be the case. Any idea what I need to start looking for? Or is this normal behaviour and the memory will be there when it is needed? Regrads, Ute ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: Session timeout and memory
On 9 Sep 2006, at 09:51, Ute Hoffmann wrote: I would expect that when a session times out that then the used memory in this session is freed imediately... but I see in my memory logging that that seems not to be the case. Any idea what I need to start looking for? Or is this normal behaviour and the memory will be there when it is needed? Succinctly: garbage collection. Don't worry about it. Paul ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
