On Tue, Feb 09, 2016 at 05:07:45AM +0000, Michael Still wrote: > On Tue, Feb 9, 2016 at 4:59 AM, Joshua Hesketh <joshua.hesk...@gmail.com> > wrote: > > > On Thu, Feb 4, 2016 at 2:44 AM, James E. Blair <cor...@inaugust.com> > > wrote: > >> > >> On the subject of clearing the cache more often, I think we may not want > >> to wipe out the cache more often than we do now -- in fact, I think we > >> may want to look into ways to keep from doing even that, because > >> whenever we reload now, Zuul slows down considerably as it has to query > >> Gerrit again for all of the data previously in its cache. > >> > > > > I can see a lot of 3rd parties or simpler CI's not needing to reload zuul > > very often so this cache would never get cleared. Perhaps cached objects > > should have an expiry time (of a day or so) and can be cleaned up > > periodically? Additionally if clearing the cache on a reload is causing > > pain maybe we should move the cache into the scheduler and keep it between > > reloads? > > > > Do you guys use oslo at all? I ask because the olso memcache stuff does > exactly this, so it should be trivial to implement if you don't mind > depending on oslo. > So, I was thinking about this the other day, what would be needed to make zuul-server more redundant. Since we can scale out merges, why not the server part too. This should help mitigate issues (eg: memory leaks, crash).
I was thinking something like memcache / redis / some other tool to externalize queues would be needed. But obviously, this adds additional overhead to running zuul. It would be interesting to see if ansible-launchers could be scaled out too over multiple hosts. Since we'd be pushing more SSH connections from zuul now. > Hope this helps, > Michael > > -- > Rackspace Australia > _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra