We can certainly add swap but imo swap is just a slower ram. At least in my experience when a system starts needing to swap things have already gone beyond recovery and swap just delays the inevitable program crash. In any case there's a few ways we can add swap if we really want to:
1) At the job level, create a swap file and activate it for the size needed for the job. 2) At the jenkins-init level (via jenkins-scripts in releng/builder) we can add swap for systems here 3) Bake it into the VM via packer file If we want to try this with a specific job type then I would try it with option 1 first and see how things go. If we find it really useful we can move it into 2 or 3. Regards, Thanh On Mon, Nov 13, 2017 at 6:19 PM, Luis Gomez <[email protected]> wrote: > +1, adding Andy: is it possible to get swap space in the VM from the cloud > provider? otherwise is it possible to modify the releng/builder packer > scripts to add the swap? > > On Nov 13, 2017, at 3:05 PM, Sam Hague <[email protected]> wrote: > > On Thu, Nov 9, 2017 at 5:45 AM, Stephen Kitt <[email protected]> wrote: > >> Exactly, that’s what I was alluding to with “the lack of swap probably >> doesn’t help” ;-). We should really configure VMs with a small amount >> of swap, it can save the kernel in tricky situations... >> > I was looking at this also and wondering about swap. We had a different > issue where the robot VMs blow up on producing the log.html file because it > ends up being a 1gb file. With swap I think it would have passed fine. As > is, we bumped the 2gb vm to a 4gb to make it work. So we will likely hit it > again. > >> >> On Thu, 9 Nov 2017 01:47:08 -0800 >> Anil Vishnoi <[email protected]> wrote: >> >> > I suspect you might hit it again if you run it a bit longer because >> > of this >> > >> > [Thu Nov 2 05:58:08 2017] Free swap = 0kB >> > [Thu Nov 2 05:58:08 2017] Total swap = 0kB >> > >> > >> > On Thu, Nov 9, 2017 at 1:39 AM, Stephen Kitt <[email protected]> wrote: >> > >> > > On Thu, 9 Nov 2017 10:28:14 +0100 >> > > Robert Varga <[email protected]> wrote: >> > > > On 02/11/17 23:02, Luis Gomez wrote: >> > > > > 1) JVM does not kill itself, the OS does instead after the java >> > > > > process grows to 3.7G in a VM of 4G RAM (note Xmx is set to 2G >> > > > > but still the jvm goes far beyond that). >> > > > >> > > > Indicates this lies out side of heap -- check thread count. >> > > >> > > We verified separately that this is an OOM issue, but one detected >> > > by the kernel rather than by the JVM (the OOM killer kills the JVM, >> > > see >> > > https://jira.opendaylight.org/secure/attachment/14207/dmesg.log.txt >> > > for details; the number of threads wasn’t an issue here, but the >> > > lack of swap probably didn’t help). >> > > >> > > Upgrading to OpenJDK 8 patch 151 fixed the problem, it might have >> > > been related to one of the several memory usage bugs in 144 that >> > > were fixed in 151. It’s probably just moving the goalposts though >> > > since the problem was new — basically, I suspect we recently >> > > started using a little too much off-heap memory for some reason, >> > > and the upgrade to 151 reduces the JVM’s memory usage enough to >> > > make us fit in our VMs again. >> > > >> > > Regards, >> > > >> > > Stephen >> >
_______________________________________________ controller-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/controller-dev
