In embedded mode the ITs were failing without the change. I can't remember what you were looking at but what problem is it causing?
jvz On 2012-12-12, at 11:35 AM, Kristian Rosenvold <[email protected]> wrote: > Does that mean m2e ditches the container every now and then? > > In that case the whole unloading jason implemented can be reverted...? > > K > > Den 12. des. 2012 kl. 16:44 skrev Igor Fedorenko <[email protected]>: > >> For tests, I think the easiest is to check number of realms at the end >> of each test and drop plexus container if it grew over certain number of >> realms. Pick the number large enough to fit in 128M of permgen and I >> think this will provide good tradeoff between performance and memory >> usage. This does mean separate container per each test thread, but I >> think this is required for other reasons. >> >> For IDEs, I would not try to guess what they need and let IDE developers >> come with proposed improvements to caching. I can't speak for other >> IDEs, but m2e for example, already deals with project realms properly >> and plugin realms are not usually an issue, so we don't really have any >> issues we need to fix. >> >> -- >> Regards, >> Igor >> >> On 2012-12-12 3:31 AM, Kristian Rosenvold wrote: >>> 2012/12/12 Milos Kleint <[email protected]>: >>>> how much memory will be freed by the soft reference? if it's not a big >>>> chunk, most likely not worth it, soft references are released only >>>> when your VM is really, really in trouble. by the time it gets >>>> released, you've been slowed down by repeatedly hitting the ceiling of >>>> your memory and CGed frequently. >>> >>> I think the appropriate question to ask is "how much memory would be >>> freed by an eviction strategy?" and maybe "who needs it?". >>> >>> The answer to "how much" can precisely be formulated as "a truckload". >>> The surefire IT's in embedded mode require 350MB permgen and 500MB >>> memory. When things are evicted those numbers are more or less halved >>> (I seem to believe they ran in 128MB permgen!). I will come up with >>> some hard numbers on that at some point... >>> >>> An embedded core used to hold on to a *lot* of classloaders, both for >>> plugins, extensions and projects. Some of these are never freed and >>> will hence leak for as long as the embedded container is being kept >>> alive. So for someone IDE-like embedding maven it would definitely >>> make sense to release and reallocate the embedded maven instance every >>> time a project is loaded/reloaded. Just recently it was changed to >>> always deallocate the plugins (but still leaking extension/project >>> realms), which is probably too much or too little. From an IDE point >>> of view 0.5 seconds added overhead /may/ be ok (while certainly not >>> "world class performance"). >>> >>> For something like an embedded test-run it might make sense to just >>> ditch the embedded container every N runs and start with a fresh one. >>> >>> The real problem starts when you decide to hold onto the embedded instance >>> and >>> declare that the instance should release class realms according to >>> some eviction strategy. Like so many times before, neither >>> softreferences nor weakreferences really seem appropriate; one is too >>> liberal in letting go, the other too conservative. Currently core does >>> not track actual usage of each class realm (although it would be >>> fairly simple to implement) ; if it did we could do all sorts of cool >>> stuff. >>> >>> I prefer a /predictable/ eviction strategy. We *could* do something like >>> a round-robin eviction (evict a rotating 25% of all classrealms every >>> time) or a usage based eviction (evict all unused and every N usages >>> of a given class realm). But then again maybe this whole issue is best >>> solved by restoring the old strategy of "no eviction" and tell the >>> clients to evict us every now and then. That would work very well for >>> embedded test runs, unsure what something like m2e would do about >>> it.... >>> >>> Kristian >>> >>> >>> >>> >>> >>> >>> >>> >>>> >>>> just an illustrative story, most likely not related to this usecase >>>> but still interesting >>>> while embedding maven in netbeans, we've SoftReferenced MavenProject >>>> at some point in time. MavenProjects can hold a big object tree, >>>> mostly via the cache in ProjectBuildingRequest. So once you run out of >>>> memory, the SRs get dropped however sometimes the MP instances are >>>> again immediately required, so they are loaded again. And dropped. And >>>> loaded.... and the IDE grinds to a halt. In some cases, one gets OOME >>>> fairly fast, but in others it can take fairly long. >>>> >>>> BTW we don't embed maven builds, just MavenProject loading in netbeans >>>> >>>> Milos >>>> >>>> On Sun, Dec 9, 2012 at 11:04 PM, Christian Schulte <[email protected]> wrote: >>>>> Am 12/09/12 22:58, schrieb Kristian Rosenvold: >>>>>> >>>>>> Anyone else have any ideas about eviction strategies ? >>>>> >>>>> Without having looked at the code. GC driven by using soft references. >>>>> >>>>> -- >>>>> Christian >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
