Re: Memory Management

2009-03-09 Thread Chuck Hill
On Mar 6, 2009, at 6:08 PM, Jeff Schmitz wrote: On Mar 6, 2009, at 7:05 PM, Chuck Hill wrote: from the Entry table, the Game table and the TeamPopup table that are associated with a single pool via joins from the pool's entries to each entry's games to each game's teams. To process 1 pool

Re: Memory Management

2009-03-06 Thread Jeff Schmitz
I think I found the answer in a question you answered back in 2006... prefetchingRelationshipKeyPaths is about pulling in all objects related to the object(s) being fetched. If you fetch the person and set prefetchingRelationshipKeyPaths to ("personNames", "personNames.publications") EOF w

Re: Memory Management

2009-03-06 Thread Jeff Schmitz
On Mar 6, 2009, at 7:05 PM, Chuck Hill wrote: That is _batch_ fetching, not _pre_fetching. They do similar things, but prefetching is easier to use for most cases. I'm working on the prefetching and was wondering, is there any way to prefetch several "layers" of relationships into the edi

Re: Memory Management

2009-03-06 Thread Jeff Schmitz
On Mar 6, 2009, at 7:05 PM, Chuck Hill wrote: from the Entry table, the Game table and the TeamPopup table that are associated with a single pool via joins from the pool's entries to each entry's games to each game's teams. To process 1 pool that has say 300 entries, each entry having 63

Re: Memory Management

2009-03-06 Thread Chuck Hill
On Mar 6, 2009, at 4:57 PM, Jeff Schmitz wrote: On Mar 6, 2009, at 6:19 PM, Chuck Hill wrote: On Mar 4, 2009, at 6:30 PM, Jeff Schmitz wrote: A little more info on my problem (yes I'm confused). For my processing, all my data is not in one table. I need to work on one coherent set o

Re: Memory Management

2009-03-06 Thread Jeff Schmitz
On Mar 6, 2009, at 6:19 PM, Chuck Hill wrote: On Mar 4, 2009, at 6:30 PM, Jeff Schmitz wrote: A little more info on my problem (yes I'm confused). For my processing, all my data is not in one table. I need to work on one coherent set of data at a time that is related across tables via

Re: Memory Management

2009-03-06 Thread Chuck Hill
On Mar 4, 2009, at 6:30 PM, Jeff Schmitz wrote: A little more info on my problem (yes I'm confused). For my processing, all my data is not in one table. I need to work on one coherent set of data at a time that is related across tables via joins. My tables are setup as follows: Pool-

Re: Memory Management

2009-03-04 Thread Jeff Schmitz
A little more info on my problem (yes I'm confused). For my processing, all my data is not in one table. I need to work on one coherent set of data at a time that is related across tables via joins. My tables are setup as follows: Pool*>>Entry-63>>Game-2>TeamPopup To do my pr

Re: Memory Management

2009-03-04 Thread Chuck Hill
/display/WO/EOF-Using+EOF-EOF+Performance+Tuning http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+Management http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Bulk+Operations Chuck On Sunday, March 01, 2009, at 09:55PM, "Lachlan Deck" > wrote:

Re: Memory Management

2009-03-04 Thread Jeff Schmitz
I am actually using ERXBatchFetchUtilities. e.g. ERXBatchFetchUtilities.batchFetch(pool.entries(), Entry.GAME.append(Game.TEAM_POPUPS)); Is that the same thing, or something different? Is raw row fetching likely to be even faster? Thanks! Jeff On Sunday, March 01, 2009, at 09:55PM, "

Re: Memory Management

2009-03-04 Thread Jeff Schmitz
for the threaded process that has its own object store? Is just using ec.reset() at key points the way to go for the ec's? What about for the object stores? http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+Management Your best bet (IMO) is to unlock and dispose of

Re: Memory Management

2009-03-02 Thread Chuck Hill
confluence/display/WO/EOF-Using+EOF-Memory+Management Your best bet (IMO) is to unlock and dispose of the EC and then create a new one and carry on. Look after the editing contexts, and the object stores will take care of themselves. Chuck -- Chuck Hill Senior Consultant /

Re: Memory Management

2009-03-02 Thread Jeff Schmitz
ct store? Is just using ec.reset() at key points the way to go for the ec's? What about for the object stores? http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+Management Your best bet (IMO) is to unlock and dispose of the EC and then create a new one and carry on

Re: Memory Management

2009-03-02 Thread Andrew Lindesay
Hello Chuck; I can only guess that it is comparing snapshots' keys to find the correct snapshot to "replace" from the inbound data (from the fetch) in a map's bucket which is quite large. cheers. Comparing snapshots would not be for uniquing. Unique tests would be some variation of snap

Re: Memory Management

2009-03-02 Thread Chuck Hill
On Mar 2, 2009, at 1:18 PM, Andrew Lindesay wrote: Hello Chuck; A big part of slowing-down with saturated object stores is that as there are more and more EO's in memory it takes longer and longer for fetches to be processed because as the system ensures uniqueness, it will have to keep

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Mr. G Brown
Just recently, I answered a question about value types without recalling, or mentioning, that this is documented there also (at http://wiki.objectstyle.org/confluence/display/WO/EOF-Modeling- Common+Pitfalls+and+Troubleshooting) , although that page name might not make it extremely easy to fi

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Pascal Robert
Le 09-03-02 à 15:24, Chuck Hill a écrit : On Mar 2, 2009, at 9:03 AM, Ray Kiddy wrote: Is there a reason we do not seem to be talking about or updating the wiki documentation anymore? Yes: - loss of Will to Live - realization that only .01% of people will actually look at the wiki bef

Re: Memory Management

2009-03-02 Thread Andrew Lindesay
Hello Chuck; A big part of slowing-down with saturated object stores is that as there are more and more EO's in memory it takes longer and longer for fetches to be processed because as the system ensures uniqueness, it will have to keep checking to see if it already has each new row of da

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Chuck Hill
On Mar 2, 2009, at 12:38 PM, Lachlan Deck wrote: On 03/03/2009, at 7:24 AM, Chuck Hill wrote: On Mar 2, 2009, at 9:03 AM, Ray Kiddy wrote: Looking at this post, I realized there could be a discussion of this on the doc (at http://wiki.objectstyle.org/confluence/display/WO/Programming__WebO

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Lachlan Deck
Woops .. meant to cc list. On 03/03/2009, at 7:38 AM, Lachlan Deck wrote: On 03/03/2009, at 7:24 AM, Chuck Hill wrote: On Mar 2, 2009, at 9:03 AM, Ray Kiddy wrote: Looking at this post, I realized there could be a discussion of this on the doc (at http://wiki.objectstyle.org/confluence/dis

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Chuck Hill
On Mar 2, 2009, at 9:03 AM, Ray Kiddy wrote: Is there a reason we do not seem to be talking about or updating the wiki documentation anymore? Yes: - loss of Will to Live - realization that only .01% of people will actually look at the wiki before posting - realization that only .005% of

Re: Memory Management

2009-03-02 Thread Chuck Hill
On Mar 1, 2009, at 7:14 PM, Andrew Lindesay wrote: Hello Jeff; I posted on this as well? Did you get this post and have you tried it? Load the list of EO's into memory to be processed as raw-rows with the PK in the raw rows. Break that lists up into batches of, for example, 100 items

Re: Memory Management

2009-03-02 Thread Chuck Hill
out for the object stores? http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+Management Your best bet (IMO) is to unlock and dispose of the EC and then create a new one and carry on. Look after the editing contexts, and the object stores will take care of themselve

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Andrew Lindesay
Hello Ray; Yes good point; I'll add it to my to-do list. cheers. Is there a reason we do not seem to be talking about or updating the wiki documentation anymore? ___ Andrew Lindesay www.lindesay.co.nz ___ Do not post admin requests to the list. T

Re: Memory Management (or rather documenting thereof...)

2009-03-02 Thread Ray Kiddy
Is there a reason we do not seem to be talking about or updating the wiki documentation anymore? Looking at this post, I realized there could be a discussion of this on the doc (at http://wiki.objectstyle.org/confluence/display/WO/Programming__WebObjects-EOF-Modeling-Fetch+Specifications) a

Re: Memory Management

2009-03-01 Thread Andrew Lindesay
What a sneaky editing context -- I didn't even notice sans the locking code. Thanks! cheers. You can give it a new ec on each iteration as I showed in my example code :-) ... I think Jeff may have the same problem with that class because it keeps using the same EC -- I may be mistaken si

Re: Memory Management

2009-03-01 Thread Jeff Schmitz
Thanks! A little late to try tonight, but since you laid it all out like that I'll give it a go. Jeff On Mar 1, 2009, at 10:04 PM, Andrew Lindesay wrote: Hello Jeff; Fair enough. I will keep it simple... First, create a fetch specification for the main EO that you wish to operate on an

Re: Memory Management

2009-03-01 Thread Lachlan Deck
You can give it a new ec on each iteration as I showed in my example code :-) On 02/03/2009, at 3:06 PM, Andrew Lindesay wrote: I think Jeff may have the same problem with that class because it keeps using the same EC -- I may be mistaken since I don't use this class myself! cheers. Now

Re: Memory Management

2009-03-01 Thread Andrew Lindesay
Hello Lachlan; I think Jeff may have the same problem with that class because it keeps using the same EC -- I may be mistaken since I don't use this class myself! cheers. Now .. you might also want to have a look at Wonder's ERXFetchSpecificationBatchIterator. ___ Andrew Lindesay www.l

Re: Memory Management

2009-03-01 Thread Andrew Lindesay
Hello Jeff; Fair enough. I will keep it simple... First, create a fetch specification for the main EO that you wish to operate on and prep it for pulling out the primary key; EOEntity fooE = EOModelGroup.globalModelGroup().entityNamed("Foo"); EOFetchSpecification fs = ...

Re: Memory Management

2009-03-01 Thread Lachlan Deck
On 02/03/2009, at 2:30 PM, Jeff Schmitz wrote: Maybe it's not as hard as it sounds? Keep in mind that right now when you say "Load the list of EO's into memory to be processed as raw-rows with the PK in the raw rows." I have NO idea what you're talking about. When you set the flag on th

Re: Memory Management

2009-03-01 Thread Jeff Schmitz
Hi Andrew, Yes, I saw your post. However, I'm not an advanced EOF user, nor do I know SQL very well (it's one of the reason I like webobjects). That and the fact that any change I make needs to be working in a couple of weeks makes me want to save a change like you suggest for later w

Re: Memory Management

2009-03-01 Thread Andrew Lindesay
Hello Jeff; I posted on this as well? Did you get this post and have you tried it? Load the list of EO's into memory to be processed as raw-rows with the PK in the raw rows. Break that lists up into batches of, for example, 100 items. For each bunch of 100, create a new EC, lock it and th

Re: Memory Management

2009-03-01 Thread Jeff Schmitz
using ec.reset() at key points the way to go for the ec's? What about for the object stores? http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+Management Your best bet (IMO) is to unlock and dispose of the EC and then create a new one and carry on. Look after the e

Re: Memory Management

2009-02-24 Thread Andrew Lindesay
Hello Jeff; Load the list of EO's into memory to be processed as raw-rows with the PK in the raw rows. Break that lists up into batches of, for example, 100 items. For each bunch of 100, create a new EC, lock it and then fetch those EO's into memory and use pre-fetching to optimise the

Re: Memory Management

2009-02-24 Thread Ray Kiddy
On Feb 23, 2009, at 9:13 PM, Jeff Schmitz wrote: Hello, In my app I frequently need to run operations that read in nearly all the data in my database. One I run in the foreground (I have concurrent request handling on), and another I run in the background with its own EOObjectStoreCo

Re: Memory Management

2009-02-24 Thread Chuck Hill
I know my each process is through with them, especially for the threaded process that has its own object store? Is just using ec.reset() at key points the way to go for the ec's? What about for the object stores? http://wiki.objectstyle.org/confluence/display/WO/EOF-Using+EOF-Memory+M

Memory Management

2009-02-23 Thread Jeff Schmitz
Hello, In my app I frequently need to run operations that read in nearly all the data in my database. One I run in the foreground (I have concurrent request handling on), and another I run in the background with its own EOObjectStoreCoordinator. If I understand how things work (and t

Re: WO and Memory Management

2006-07-16 Thread shaun
Chuck Hill wrote: Hi Owen, Comments in-line below. On Jul 13, 2006, at 10:11 PM, Owen McKerrow wrote: Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and discovered something strange. Their comments

Re: WO and Memory Management

2006-07-14 Thread Guido Neitzer
On 14.07.2006, at 20:01 Uhr, Chuck Hill wrote: Wonder has a display group or a fetch iterator (my memory is a bit foggy this morning), that will allow you to fetch in batches of records without hauling in the entire result set at once. This is very useful if you don't need all the results

Re: WO and Memory Management

2006-07-14 Thread Chuck Hill
On Jul 14, 2006, at 10:03 AM, John Larson wrote: Either all of your actions are a little slow or some are very slow. Sight unseen, my money is on the latter. Usually this is the result of DB access, either queries that are slow to evaluate due to missing indexes DOH ! **Hits self in th

Re: WO and Memory Management

2006-07-14 Thread Chuck Hill
Hi Owen, Comments in-line below. On Jul 13, 2006, at 10:11 PM, Owen McKerrow wrote: Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and discovered something strange. Their comments The webobjects

Re: WO and Memory Management

2006-07-14 Thread John Larson
Either all of your actions are a little slow or some are very slow. Sight unseen, my money is on the latter. Usually this is the result of DB access, either queries that are slow to evaluate due to missing indexes DOH ! **Hits self in the head** Thats one thing I need to do when ther

Re: WO and Memory Management

2006-07-14 Thread Chuck Hill
On Jul 13, 2006, at 10:37 PM, Owen McKerrow wrote: Yes a couple of days ago we were. Although we seem to be past them now and are dealing with the issue of WO not appearing to let go of the memory it uses and it having "broken pipe" issues. It is not WO that is not letting go of the memo

Re: WO and Memory Management

2006-07-14 Thread Mike Schrag
You can get a query execution plan from Oracle and see exactly which indexes are and are not being used (with the "explain plan" command -- it requires a little setup first).  You can also install statspack (I think this is renamed now?) and get extensive debugging stats from it.On Jul 14, 2006, at

Re: WO and Memory Management

2006-07-14 Thread Dov Rosenberg
If you are moving to Oracle and you have bulk loaded a bunch of data you should probably analyze the schema and indexes - otherwise the oracle optimizer doesn't make good decisions. Dov Rosenberg Inquira Corporation The Knowledgebase experts http://www.inquira.com On 7/14/06 1:11 AM, "Owen McKer

Re: WO and Memory Management

2006-07-14 Thread Art Isbell
On Jul 13, 2006, at 7:37 PM, Owen McKerrow wrote: Although we seem to be past them now and are dealing with the issue of WO not appearing to let go of the memory it uses and it having "broken pipe" issues. Broken pipe messages and "No instance available" responses can be symptoms of resp

Re: WO and Memory Management

2006-07-13 Thread Owen McKerrow
Yes a couple of days ago we were. Although we seem to be past them now and are dealing with the issue of WO not appearing to let go of the memory it uses and it having "broken pipe" issues. Owen McKerrow WebMaster, emlab Ph : +61 02 4221 5517 http://emlab.uow.edu.au - - - - - - - - - - - -

Re: WO and Memory Management

2006-07-13 Thread apl
Hello Owen; its better to explicitly call System.gc() in WO App's. Whats the rule of thumb here in terms of how often, is there a better spot than others etc etc ? It is only ever a hint, never a demand. I have resorted to this infrequently. That's correct, but I think the problem arise

Re: WO and Memory Management

2006-07-13 Thread Owen McKerrow
Hi Chuck Again see comments below. Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and discovered something strange. Their comments The webobjects versions have been stress tested further, we

Re: WO and Memory Management

2006-07-13 Thread Chuck Hill
Hi Owen, On Jul 13, 2006, at 7:33 PM, Owen McKerrow wrote: Thanks for the response Chuck. See comments below On 14/07/2006, at 11:53 AM, Chuck Hill wrote: On Jul 13, 2006, at 6:15 PM, Owen McKerrow wrote: Hi All, We are setting up an application of ours on a clients server, they d

Re: WO and Memory Management

2006-07-13 Thread Owen McKerrow
Thanks for the response Chuck. See comments below On 14/07/2006, at 11:53 AM, Chuck Hill wrote: On Jul 13, 2006, at 6:15 PM, Owen McKerrow wrote: Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and

Re: WO and Memory Management

2006-07-13 Thread Chuck Hill
On Jul 13, 2006, at 6:15 PM, Owen McKerrow wrote: Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and discovered something strange. Their comments The webobjects versions have been stress tested fu

WO and Memory Management

2006-07-13 Thread Owen McKerrow
Hi All, We are setting up an application of ours on a clients server, they did some stress testing to see if they needed a bigger box and discovered something strange. Their comments The webobjects versions have been stress tested further, we are getting ok results but far worse then