2012/9/13 Maik Musall <[email protected]> > Hi Alex, Hi Chuck, > > Am 13.09.2012 um 02:28 schrieb Chuck Hill <[email protected]>: > >>> Never out of memory. The app is allowed to grow up to 24 GByte, stays > in the 1-4 GByte range in normal use and occasionally grows up to 12 GByte > with the most advanced statistics that tend to suck in the whole database. > >>> > >>> That's also the reason though that I can't use separate EOF stacks for > the statistics, because as soon as there were more than one of them, I'd > have multiple 10 GByte-ish snapshot caches. The server has 48 GByte and I > don't really want to hit that limit... and with separate stacks, it also > would be difficult to keep the stats reflect current changes in the other > stacks. > >> > >> > >> I am not sure about the background threads (depends on how long OSC > locks are held), but using ECs sharing the same EOF stack with regular > requests is likely to cause problems like you are seeing. > >> > >> Do you mean that the application would be unresponsive while the lock > was held in the background thread, or that simply doing it that way will > lead to unrecoverable deadlocks? > >> If you do massive fetches in the background, that will block other > requests as the only OSC is locked. > > > > Correct. > > > > > >> That said, I think (and correct me if I'm wrong) if you lock the ec but > do not fetch anything with this ec, other ecs can still access the db. > > > > Also correct. The lock contention is only when fetching or saving. It > can also happen if your code (or Wonder code that you are using) locks > something in EOControl or EOAccess. > > > I'm very familiar with that stuff, and my users know how it feels to wait > for that lock :-) > > >> Anyway, the best practice is to use a dedicated OSC to do background > work. > >> Maik, you should use a dedicated OSC for your stats, and try, if > possible to clean memory, for example : > >> > >> ec.lock(); > >> try { > >> // huge loop to compute stats > >> for (i = 0; i < 1000000; i++) { > >> // doing stuff with ec... > >> // cycling the ec > >> if (i % 100 == 0) { > >> ec.unlock(); > >> ec.dispose(); > >> ec = newEditingContextForMyWork(); > >> ec.lock(); > >> } > >> } > >> } finally { > >> ec.unlock(); > >> } > > > > If practical (I recall that it is not in Maik's case) that can be a good > way of limiting memory usage. > > Right, not practical for me. I even rely on those statistics to fill the > snapshot cache with data that other users will need in a minute anyway to > speed up overall response times. Those statistics are not strictly > background processes, they are user interaction that happens to be > implemented in a worker thread while the user is displayed a long response > page. > > What I've done to improve concurrent response times while those stats > fetch their 300000 EOs: I fetch them in batches of a few 1000 and release > the lock in between. This is the method I can call on my manual-locking > editing context between batches: > > public void shortLockRelease() { > unlock(); > try { > Thread.sleep( 50 ); > } catch( InterruptedException e ) { > e.printStackTrace(); > } finally { > lock(); > } > } > > This effectively gives other threads the opportunity to sneak in a few > transactions before the stats worker resumes grabbing the OSC's resources, > and is enough to keep response times within a reasonable limit. Users feel > it when stats are running, but they don't have to really wait any more. > I've even tuned those 50 ms. Less than that and don't get the desired > effect. More than that and you needlessly increase the stats execution time.
Interesting setup, thanks for sharing, looks like one giant VM (and EOF) can handle this amount of objects ! If the DB is touched by this app only, you could fetch all the stats at startup... but I imagine this is more complicated :) Alex
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [email protected]
