On Wed, Jun 09, 2021 at 04:36:25PM +0200, Julian Andres Klode wrote:
> Hi all,
> 
> we are currently severily limited by having turned off all arm64
> bos01 workers, and now bos01 networking being broken.
> 
> I disabled bos01 arm64 workers on May 27, because it was a bit
> unstable, and xnox wanted to run kernel tests on bos02 only,
> and the queues were empty. Last week, queues got filled again,
> and we then lagged behind a lot. We know that machines in bos01
> have crashing qemus on reboot, as well as failures to boot with
> init failing with exitcode=0x0005 or something like that.

Are there bugs filed for these issues someone could point to? I've
been debugging an intermittent crash that looks like an upstream shim
bug (LP: #1928010), but I don't think I've seen the QEMU crash/init
failures.

 -dann

> Yesterday, we started one instance again, but unfortunately
> discovered that networking is completely broken, so the instance
> has yet to actually run any tests for us to know how unstable
> the bos01 cloud is.
> 
> I think the way things are going, we should reenable all bos01
> workers ASAP once the networking works again, even if they're
> super flaky, as we do seem to be limiting our resources as
> shown by the difference in how fast we reduce the queue :D
> 
> We need to get the infrastructure issues resolved too, though,
> to make sure tests actually work.
> 

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Reply via email to