Thanks Dan. Do you happen to have observed the memory trend during a build?
After a couple more attempts it passed the build once, so that shows it's possible to pass.. but even though it's a small sample so far that's 1 pass vs 3 OOMs on my machine. Even the one time it successfully completed the tests I see it wasted ~80% of total build time doing GC runs.. it was likely very close to fall over, and definitely not an efficient setting for regular builds. Observing trends on my machine I'd guess a reasonable value to be around 5GB to keep builds fast, or a minimum of 1.3 GB to be able to complete successfully without often failing. The memory issues are worse towards the end of the testsuite, and steadily growing. I won't be able to investigate further as I need to urgently work on modules, but I noticed there are quite some MBeans according to JConsole. I guess it would be good to check if we're not leaking the MBean registration, and therefore leaking (stopped?) CacheManagers from there? Even near the beginning of the tests, when forcing a full GC I see about 400MB being "not free". That's quite a lot for some simple tests, no? Thanks, Sanne On 15 February 2018 at 06:51, Dan Berindei <dan.berin...@gmail.com> wrote: > forkJvmArgs used to be "-Xmx2G" before ISPN-8478. I reduced the heap to 1G > because we were trying to run the build on agent VMs with only 4GB of RAM, > and the 2GB heap was making the build run out of native memory. > > I've yet to see an OOME in the core tests, locally or in CI. But I also > included -XX:+HeapDumpOnOutOfMemoryError in forkJvmArgs, so assuming there's > a new leak it should be easy to track down in the heap dump. > > Cheers > Dan > > > On Wed, Feb 14, 2018 at 11:46 PM, Sanne Grinovero <sa...@infinispan.org> > wrote: >> >> Hey all, >> >> I'm having OOMs running the tests of infinispan-core. >> >> Initially I thought it was related to limits and security as that's >> the usual suspect, but no it's really just not enough memory :) >> >> Found that the root pom.xml sets a <forkJvmArgs> property to Xmx1G for >> surefire; I've been observing the growth of heap usage in JConsole and >> it's clearly not enough. >> >> What surprises me is that - as an occasional tester - I shouldn't be >> the one to notice such a new requirement first. A leak which only >> manifests in certain conditions? >> >> What do others observe? >> >> FWIW, I'm running it with 8G heap now and it's working much better; >> still a couple of failures but at least they're not OOM related. >> >> Thanks, >> Sanne >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev