On 19 August 2016 at 00:07, Sagi Shnaidman <sshna...@redhat.com> wrote: > Hi, > > we have a problem again with not enough memory in HA jobs, all of them > constantly fails in CI: http://status-tripleoci.rhcloud.com/
Have we any idea why we need more memory all of a sudden? For months the overcloud nodes have had 5G of RAM, then last week[1] we bumped it too 5.5G now we need it bumped too 6G. If a new service has been added that is needed on the overcloud then bumping to 6G is expected and probably the correct answer but I'd like to see us avoiding blindly increasing the resources each time we see out of memory errors without investigating if there was a regression causing something to start hogging memory. Sorry if it seems like I'm being picky about this (I seem to resist these bumps every time they come up) but there are two good reasons to avoid this if possible o at peak we are currently configured to run 75 simultaneous jobs (although we probably don't reach that at the moment), and each HA job has 5 baremetal nodes so bumping from 5G too 6G increases the amount of RAM ci can use at peak by 375G o When we bump the RAM usage of baremetal nodes from 5G too 6G what we're actually doing is increasing the minimum requirements for developers from 28G(or whatever the number is now) too 32G So before we bump the number can we just check first if its justified, as I've watched this number increase from 2G since we started running tripleo-ci thanks, Derek. [1] - https://review.openstack.org/#/c/353655/ > I've created a patch that will increase it[1], but we need to increase it > right now on rh1. > I can't do it now, because unfortunately I'll not be able to watch this if > it works and no problems appear. > TripleO CI cloud admins, please increase the memory for baremetal flavor on > rh1 tomorrow (to 6144?). > > Thanks > > [1] https://review.openstack.org/#/c/357532/ > -- > Best regards > Sagi Shnaidman __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
