Re: java.lang.outofmemory
On 8 Mar 2006, at 00:08, Ken Anderson wrote: I used to think that too - anyone from the old days do :) Art proved me wrong though... Sounds like my head is stuck in the pre-5.2 memory model, which I need to fix. I'll need to compare against the docs I quoted, and find out if the inconsistencies are in me or them. I can see some test apps in my near future. Thanks to everyone that corrected me - Art, Chuck, Mike, and, of course, Ken. Paul ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On 7 Mar 2006, at 23:10, Randy Wigginton wrote: At the risk of appearing foolish to a lot of people… where do I adjust the memory setting in XCode? I’ve gone to the target, and pasted the command line changes into the “pure java” field. But my startup parameters don’t change. Where does it really belong? If you set it per instance using JavaMonitor, it goes in Instance Settings, Additional Arguments, with all the other command line arguments. In the project (Xcode 2.2), inspect the Executable for your app, and use the Arguments tab. Like the example Jacky Gagnon used: -Xms64m -Xmx300m Paul ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
java.lang.outofmemory
Hello,
My program loads a great deal of data from the database. I
process it, then attempt to free the memory. My code looks
vaguely like this:
For (all objects) {
EOEditingContext tmp = new EOEditingContext();
load objects from tmp with a fetch specification
tmp.invalidateAllObjects();
tmp = null;
System.gc();
}
Even with this, Im running out of memory. Any hints
on making EO throw away everything it knows about?
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
You can increase the memory in the Monitor setup as well as in the
Application where you can add some more memory in the Java Heap which
will help in the memory management.
On 3/7/06, Randy Wigginton [EMAIL PROTECTED] wrote:
Hello,
My program loads a great deal of data from the database. I process it, then
attempt to free the memory. My code looks vaguely like this:
For (all objects) {
EOEditingContext tmp = new EOEditingContext();
load objects from tmp with a fetch specification
tmp.invalidateAllObjects();
tmp = null;
System.gc();
}
Even with this, I'm running out of memory. Any hints on making EO throw
away everything it knows about?
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/tanmoy.roy%40gmail.com
This email sent to [EMAIL PROTECTED]
--
Best,
Tanmoy
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
There are a number of things that can be done to resolve this.1. Allow your application to use more memory by increasing the heap size. [This is good if you are running your app with too little heap space]2. Make your application use less memory by qualifying your fetches more precisely. [May not be possible in some cases]3. Make your application more efficient by using "Raw Row Fetching" [Uses less memory than "real" objects] [Makes your code database specific in some cases, but allows "real" fetch limits] [Can still make "real" enterprise objects from raw rows when needed]On Mar 7, 2006, at 5:10 PM, Randy Wigginton wrote: Hello,My program loads a great deal of data from the database. I process it, then attempt to “free” the memory. My code looks vaguely like this:For (all objects) {EOEditingContext tmp = new EOEditingContext();load objects from tmp with a fetch specification tmp.invalidateAllObjects();tmp = null;System.gc();} Even with this, I’m running out of memory. Any hints on making EO throw away everything it knows about? ___Do not post admin requests to the list. They will be ignored.Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)Help/Unsubscribe/Update your Subscription:http://lists.apple.com/mailman/options/webobjects-dev/robertwalker1%40mac.comThis email sent to [EMAIL PROTECTED] Robert Walker[EMAIL PROTECTED] ___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On 7 Mar 2006, at 22:46, Jacky Gagnon wrote: First, its not a good idea to use invalidateAllObjects() to free memory; call saveChanges more frequently or just fetch what you need. saveChanges() doesn't release snapshots, unfortunately, which I would surmise is the problem. If you just want to get rid of pending changes, then saveChanges() or revert() are both suitable ways of doing this; it may also be acceptable to use dispose(), which will release any EOs in the editing context - but invalidateAllObjects(), and its variants, are the only way to flush the snapshots. Expanding the java VM size is a good alternative. Paul ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
RE: java.lang.outofmemory
At the risk of appearing foolish to a lot
of people where do I adjust the memory setting in XCode? Ive gone to the
target, and pasted the command line changes into the pure java field. But my
startup parameters dont change. Where does it really belong?
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Robert Walker
Sent: Tuesday, March 07, 2006 5:58
PM
To: WebObjects Dev Apple
Subject: Re: java.lang.outofmemory
There are a number of things that can be done to resolve this.
1. Allow your application to use more memory by increasing the heap
size.
[This is good if you are running your app with too little
heap space]
2. Make your application use less memory by qualifying your fetches
more precisely.
[May not be possible in some cases]
3. Make your application more efficient by using Raw Row
Fetching
[Uses less memory than real objects]
[Makes your code database specific in some cases,
but allows real fetch limits]
[Can still make real enterprise objects
from raw rows when needed]
On Mar 7, 2006, at 5:10 PM, Randy Wigginton
wrote:
Hello,
My program
loads a great deal of data from the database. I process it, then attempt
to free the memory. My code looks vaguely like this:
For (all
objects) {
EOEditingContext
tmp = new EOEditingContext();
load
objects from tmp with a fetch specification
tmp.invalidateAllObjects();
tmp = null;
System.gc();
}
Even with
this, Im running out of memory. Any hints on making EO throw away
everything it knows about?
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list
(Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/robertwalker1%40mac.com
This email sent to [EMAIL PROTECTED]
Robert Walker
[EMAIL PROTECTED]
___
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On Mar 7, 2006, at 1:00 PM, Paul Lynch wrote: On 7 Mar 2006, at 22:46, Jacky Gagnon wrote: First, its not a good idea to use invalidateAllObjects() to free memory; call saveChanges more frequently or just fetch what you need. saveChanges() doesn't release snapshots, unfortunately, which I would surmise is the problem. But if snapshot reference counting is working as advertised, dereferencing the only editing context containing references to the snapshots should release them. it may also be acceptable to use dispose(), dispose() should be invoked when an editing context is about to be garbage-collected, so sending an explicit dispose() message seems redundant. but invalidateAllObjects(), and its variants, are the only way to flush the snapshots. Other than removing any references which doesn't involve the invalidateAllObjects() overhead of converting all of the objects to faults. So invoking invalidateAllObjects() seems like a waste to me. Aloha, Art ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On Mar 7, 2006, at 1:39 PM, Paul Lynch wrote: Snapshots are stored at a lower level than editing context, so they won't be released. Your statement seems to contradict http://developer.apple.com/ documentation/WebObjects/WhatsNew5.2/index.html: The memory allocated for the database row-level snapshots that corresponds to garbage-collected EOEnterpriseObjects is released some time after the EOEnterpriseObjects have been garbage collected. Although generally unnecessary, you can use processRecentChanges to force an EOEditingContext to decrement the snapshot reference count on those snapshots that are no longer needed. EODatabase still holds strong references to row-level snapshots and maintains a reference count for each row and its associated EOGlobalID object. I forget where they are stored, but it's possibly the parent object store. EODatabase If you're using the default object store strategy, then all sessions will be sharing the same parent, and snapshots may be retained. The only snapshots retained should be those referenced by objects in editing contexts other than the one being dereferenced after the fetched objects have been processed. But those objects fetched into the temporary editing context being used in this case and that haven't been fetched into any other editing context should be free when their editing context is dereferenced (well, when Java's garbage collector gets around to it). After these objects are freed, their snapshots should be free (again, when Java's garbage collector gets around to it). Aloha, Art ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On 8/03/2006, at 12:54 PM, Art Isbell wrote:Your statement seems to contradict On 8/03/2006, at 1:08 PM, Ken Anderson wrote:I used to think that too ... Art proved me wrong though...Hi guysThis has been an interesting discussion as it touches on area that has caused me some grief - outOfMemory problems are hard to solve, partly because they tend to fail to record all the usual evidence.Have you reached an agreement? Each new email seems to invalidate the previous advice (a bit like the old "how does WebObjects licensing work" discussion :-) )It would really nice if one of you experts could say what combination of saveChanges(), dispose(), invalidateAllObjects(), system.gc() etc is most likely to avoid memory blowout when handling large data volumes, and then if the other experts could voice agreement ? I keep bookmarking what seems like definitive advice only to have it contradicted by a following message.regardsDenis Stanton Denis Stanton[EMAIL PROTECTED] ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On Mar 7, 2006, at 2:49 PM, Denis Stanton wrote: This has been an interesting discussion as it touches on area that has caused me some grief - outOfMemory problems are hard to solve, partly because they tend to fail to record all the usual evidence. Possibly one of the reasons this type of problem is difficult to solve is the non-deterministic manner in which Java garbage collects. In the old days (yes, I was there as well :-) when an Objective-C EO was freed, by damn, it was freed :-) None of this waiting around for the JVM to do its cleaning. So you could probably do all the right things programmatically and get nailed because you're fetching more objects before Java has finished cleaning up the old ones. It's probably best to increase the maximum Java heap size as long as doing so doesn't cause paging. Too bad the JVM has to have a maximum Java heap size. It would really nice if one of you experts could say what combination of saveChanges(), dispose(), invalidateAllObjects(), system.gc() etc is most likely to avoid memory blowout when handling large data volumes, and then if the other experts could voice agreement ? I'm no expert in Java memory management. Whenever Java throws an out of memory exception, I've just doubled the maximum heap size which has solved the problem. But I'm not fetching huge data sets. I just have a bias against using invalidateAllObjects() because, for me, it's caused more problems than it's solved. dispose() is invoked automatically, so I don't see a reason to invoke it explicitly. I've read that forcing garbage collection isn't a great idea, so I've never done so. I just use a temporary editing context (don't forget to lock it!) when I'm concerned about memory usage and need to use EO's, and dereference the editing context as soon as I've finished assuming that EOF will take care of freeing snapshots automatically. When I've written utilities that transfer data from one DB to another, I've worked at the EOAdaptor level avoiding EO creation, the overhead of EOControl, etc. Doing so is certainly faster and uses less memory, but isn't convenient if one needs to do some EO processing. Aloha, Art ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
On Mar 7, 2006, at 4:49 PM, Denis Stanton wrote: On 8/03/2006, at 12:54 PM, Art Isbell wrote: Your statement seems to contradict On 8/03/2006, at 1:08 PM, Ken Anderson wrote: I used to think that too ... Art proved me wrong though... Hi guys This has been an interesting discussion as it touches on area that has caused me some grief - outOfMemory problems are hard to solve, partly because they tend to fail to record all the usual evidence. Have you reached an agreement? Each new email seems to invalidate the previous advice (a bit like the old how does WebObjects licensing work discussion :-) ) LOL! It would really nice if one of you experts could say what combination of saveChanges(), dispose(), invalidateAllObjects(), system.gc() etc is most likely to avoid memory blowout when handling large data volumes, and then if the other experts could voice agreement ? I keep bookmarking what seems like definitive advice only to have it contradicted by a following message. Can I just muddy the waters a bit? Unless you have disabled snapshot reference counting, if there are no references from an _EO_ to a snapshot, the snapshot will in time (rather quickly in my experience) be discarded and garbage collected. EO references come from your code. EO references can also come indirectly from an EC. I don't believe that an EC maintains counted references to snapshots, but I could be mistaken in this belief. EC strong references to an EO should come from unsaved changes (updates and deletes). At some point in the past, it seemed to be that the EC was at times retaining strong references to the EO when it should not have. This may have been due to the undo manager or something else internal to the EC. I'm not sure if this was a bug, a misunderstanding on my part, or me misusing the editing context. In any event, the result of this was that snapshots were not being released when I thought they should be. Disposing of the editing context fixed / appeared to fix this problem. In terms of limiting memory usage, I make it a practice to call ec.undoManager().removeAllActions() after a successful save or after a call to revert(). I find this easier to manage than setting the undo manager to null and remembering that I need it to process deletions. I make it a practice to call dispose() on an EC if I know that it will not be used again. I have not measured how much practical benefit this has over allowing the finalizer to call dispose (), but it is an easy practice to follow and causes no harm in any event. As for invalidating objects, this is something to be avoided. I admit that I have been driven to desperation a couple of times and used this, but only with regrets and reservations. Chuck -- Coming in 2006 - an introduction to web applications using WebObjects and Xcode http://www.global-village.net/wointro Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to archive@mail-archive.com
Re: java.lang.outofmemory
This topic came up in January too -- Hopefully nobody minds me quoting myself :) This is a description of the current behavior in 5.3 that one might be able to gather if one were to, say, decompile and review the entire process -- not that i would ever do this or condone it, of course -- but if you DID, you would probably gather exactly this info, which is pretty well documented (and seems to behave to-spec) in the 5.2 release notes: As of 5.2, the way it works is: The snapshots in EODatabase have a reference count. Each editing context that fetches an EO increments the reference count. The EC holds onto that EO via a WeakReference. When the WeakReference is reclaimed, the snapshot reference count can decrease (note CAN, not IMMEDIATELY WILL -- the editing context keeps reference queue which is only processed periodically). When the count gets to zero, the database forgets the snapshot. If you have entity caching enabled, then EODatabase ignore reference count (or keeps it at 1 as a minimum) and it will not go away in a read-only scenario. If you modify any entity of that type and saveChanges in your EditingContext, a cached entity's cache will be entirely flushed. (NB: Keep this in mind, because if you are caching a large amount of data that is writable, it will NOT be very smart about updating that cache -- It's blown away with every update and then it immediately reloads the entire set of objects for that entity at the next access) If you have retainsAllRegisteredObjects enabled on your editing context, it will NOT use WeakReferences. Under this circumstance, the EO reference count is only decreased when 1) you dispose the editingcontext or 2) you forget or invalidate the object. When you modify an object in an editing context, the editingcontext keeps a strong reference to the objects until you saveChanges (or revert, reset, etc), at which point the strong references are cleared and the only remaining reference is the weakreference like before. If you have an undo manager enabled, it will keep a strong reference to the affected EO's as long as the undo is around. I do wonder if EC's should be using SoftReferences instead of WeakReferences ... Would seem to be more friendly to the users of those EO's. If you are using WO pre 5.2, then none of the WeakReference stuff applies, and everything is purely done with snapshot reference counting -- it should behave like retainsAllRegisteredObjects = true in 5.2. thought they should be. Disposing of the editing context fixed / appeared to fix this problem. I ran into this same thing ... I'm guessing I must have been using pre-5.2, because when this topic came up prior to January again, I was going to post about this, but thought I should write a test case and verify it and I could never get the behavior to happen that I saw re: holding onto EO's. I had a test case trying to do all sorts of wacky things with updating, inserting, etc, and they really did free up like the docs said. Go figure. easier to manage than setting the undo manager to null and remembering that I need it to process deletions. non-null undo manager for deletions is another one I could have sworn I needed, but I tried that one too and have yet to actually require it. I didn't test that one nearly as extensively, though, so maybe there are certain border cases that do? If anyone knows, I'd be curious to find out. I make it a practice to call dispose() on an EC if I know that it will not be used again. I have not measured how much practical benefit this has over allowing the finalizer to call dispose(), but it is an easy practice to follow and causes no harm in any event. The biggest thing is how long it takes the GC to get to your object. If you call dispose(), you won't hurt anything, and you immediately decrement the snapshot reference counts. If your EC is falling out of scope or you set it to null, then I guess it's probably a tossup as to who gets to it first -- GC finalizing your EC or you manually calling dispose(). I usually do it when I know I have a long process. Again, you can only help. However (referring back to a previous message in this thread), calling System.gc() is usually a bad idea, and all you're doing is requesting a GC, not ordering one. As for invalidating objects, this is something to be avoided. I admit that I have been driven to desperation a couple of times and used this, but only with regrets and reservations. Yeah -- I totally agree too .. The couple of times I resorted to this, I regretted it later, because it ended up screwing me. The biggest thing is you toss snapshots that people might be in the middle of editing, and that's going to cause really funky problems. It's a really heavy-handed way to go. So re: the original problem -- It's certainly possible that you really do just require more
