Maxim Cournoyer <maxim.courno...@gmail.com> writes: >> Before closing this bug, it would be good to understand more about how >> this happened and from that try to think if anything can be done to >> prevent similar issues in the future? >> >> At least from what I can see on the issues, the problem was introduced >> with the update to 3.98.0 [3] and then continued with the update to 3.99 >> [4]. Given the changes in 70662 were sent to guix-patches and then >> merged less than 24 hours later, I'd imagine that wasn't sufficient time >> for data.qa.guix.gnu.org to fail attempting to build nss. > > I think in [3] you meant https://issues.guix.gnu.org/70569, not #70662.
Ah, yep. > Since this was security sensitive, I built it on x86_64, tested it there > to ensure that IceCat worked as expected, had others confirmed it worked > for them on #guix then pushed. > > In the past, I've had more patience waiting for QA to build things, but > since this is not guaranteed (it sometimes never happened), it seems > reasonable to me to promptly push security fixes that were manually > built & tested and adjust for any breakage later, if there is any. I think pushing security fixes quickly is good, but this does set a precedent on architecture support (only x86_64-linux matters). For some packages (including nss in this instance), not looking at non x86_64-linux support doesn't just affect users, the data service and ci.guix.gnu.org were particularly affected, so for some packages it's important to test across the "supported" systems just to ensure the projects own tooling doesn't break. >> 3: https://issues.guix.gnu.org/70662 >> 4: https://issues.guix.gnu.org/70618 >> >> Had the changes waited for longer, then these failures should have been >> spotted by QA, I would guess that the revision might have failed to be >> processed, and if it was processed successfully, the nss failures should >> have shown up, so maybe we should start requiring [5] that not only are >> changes sent to guix-patc...@gnu.org, but that QA processes them (to >> some extent) before merging? > > I have some apprehensions about that; given the QA build farm is > somewhat under-resourced for builds, I fear security changes could be > gated for longer periods of time than is reasonable to wait. If we go > that route, I think we should dedicate more hardware first. I think that's reasonable, I have been putting time in to the hardware, but it's not been particularly easy going. The data service instances are also still stuck on hardware I'm renting as well. In terms of QA speed, there's the resources for data.qa.guix.gnu.org and there's the hardware available for the bordeaux build farm. There's also the potential to try and add more prioritisation. Currently the data service prioritises branch revisions over patches, and newer revisions more generally, but I guess it could prioritise security related things. I'm not sure how to make that happen yet though, QA can probably come up with the priorities, but I don't know how to convey that to the data service.
signature.asc
Description: PGP signature
