Excerpts from Derek Higgins's message of 2015-01-07 02:51:41 -0800: > Hi All, > I intended to bring this up at this mornings meeting but the train I > was on had no power sockets (and I had no battery) so sending to the > list instead. > > We currently run our CI with on images built for i386, we took this > decision a while back to save memory ( at the time is allowed us to move > the amount of memory required in our VMs from 4G to 2G (exactly where in > those bands the hard requirements are I don't know) > > Since then we have had to move back to 3G for the i386 VM as 2G was no > longer enough so the saving in memory is no longer as dramatic. > > Now that the difference isn't as dramatic, I propose we switch back to > amd64 (with 4G vms) in order to CI on what would be closer to a > production deployment and before making the switch wanted to throw the > idea out there for others to digest. > > This obviously would impact our capacity as we will have to reduce the > number of testenvs per testenv hosts. Our capacity (in RH1 and roughly > speaking) allows us to run about 1440 ci jobs per day. I believe we can > make the switch and still keep capacity above 1200 with a few other changes > 1. Add some more testenv hosts, we have 2 unused hosts at the moment and > we can probably take 2 of the compute nodes from the overcloud. > 2. Kill VM's at the end of each CI test (as opposed to leaving them > running until the next CI test kills them), allowing us to more > successfully overcommit on RAM > 3. maybe look into adding swap on the test env hosts, they don't > currently have any, so over committing RAM is a problem the the OOM > killer is handling from time to time (I only noticed this yesterday). > > The other benefit to doing this is that is we were to ever want to CI > images build with packages (this has come up in previous meetings) we > wouldn't need to provide i386 packages just for CI, while the rest of > the world uses the amd64.
+1 on all counts. It's also important to note that we should actually have a whole new rack of servers added to capacity soon (I think soon is about 6 months so far, but we are at least committed to it). So this would be, at worst, a temporary loss of 240 jobs per day. _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev