On Wed, 2 Dec 2020 at 18:27, Adam Williamson <adamw...@fedoraproject.org> wrote:
> On Wed, 2020-12-02 at 09:23 -0500, Ben Cotton wrote: > > > > == How To Test == > > See QA test cases : > https://fedoraproject.org/wiki/Category:CoreOS_Test_Cases > > > > We also have regular tests days, for example > > https://fedoramagazine.org/fedora-coreos-test-day/ > > So...yeah, that's not really enough to call something a Fedora Edition > :) > > I think this has a lot of the issues we had with IoT, but turned up to > 11. > > We have an entire process for releasing a thing called "Fedora" which > is based around the idea that all the bits that make up a "Fedora > release" get built, tested, and signed off together. > > IoT stretched this process a bit, because it's not actually built as > part of the same composes as all the other "Fedora" bits. So we had to > implement an entire parallel validation process just for IoT composes. > But at least it's built *the same way* as Fedora, so we could more or > less just split existing things into two paths and use them, which is > what we did. We now have both mainline and IoT composes and validation > events. Even there, the process wasn't perfect for Fedora 33; if you > look at the log of the Fedora 33 Go/No-Go meeting[1], which is supposed > to be where we go over the status of the bits-to-be-released in great > detail and decide whether to sign them off, there is precisely one line > about IoT: > > 17:58:36 <pwhalen> IoT is also covered - > https://fedoraproject.org/wiki/Test_Results:Fedora-IoT_33_RC_20201020.0_General#ISO_Install > > ...no-one else said a word about it. So yeah, we clearly don't have > this whole "releasing from multiple streams" entirely down yet. > > CoreOS is going to be the same only worse, because it's not even built > the same way as the rest of Fedora. It's not built by Pungi, we don't > get the same messages published when CoreOS builds happen (we don't get > messages published at all, IIRC), the metadata for CoreOS builds is not > in productmd[2] form like the metadata for Pungi builds, the whole > process is entirely different. > Recently messages were added when streams are updated [0][1]. I believe that other messages could be added if needed. > > So to boil this down into a representative question: when we are doing > the Fedora 34 Go/No-Go meeting in ~four months' time, how do we decide > whether to release "Fedora CoreOS 34"? > > Fedora CoreOS 34 does not exist, you have Fedora CoreOS stable, testing and next, each stream is released every 2 weeks. The Go/No-Go is based on reports coming from each stream, next stream is promoted to testing and testing promoted to stable. If any blockers are found in next or testing the content will not be promoted to stable. I think it is fairly well explained here[2] > > > What does the question "is Fedora CoreOS 34 ready to go" even mean, in > the context of how CoreOS is built and released? What set of bits will > we be deciding to ship or not ship, and how will that have been decided > and communicated? Where will we look to find the test results and > criteria on which we would base that decision? > > To maintain a fortnightly cadence there is a strong reliance on CI, every build is tested and results are inspected during the release process. Currently a release is gated on tests running on AWS, GCP and Openstack, these tests are running on a centos-ci jenkins instance which I think cannot be access without a centos account (I might be wrong), but yeah making these tests and results more transparent could be an improvement. > > Are any of these silly questions, which would indicate that our release > process is based on assumptions which need to be fundamentally re- > examined as part of this Change? > So what defines an Edition ? I think if we don't want to accept a different philosophy about release schedule and release engineering we can just close that Change proposal. How do you consider Rawhide for example ? > > All of this is stuff we could kind of handwave so long as CoreOS wasn't > an official Edition; we could *more or less* leave the CoreOS team to > decide what a CoreOS release looked like and when it would happen and > when it was good enough to ship, and so on. > So this change comes down to Can we have a Fedora Edition that has a different workflow ? > But if we're going to call it an Edition, which is one of the primary > things that defines what Fedora *is*, I don't think we can do that any > more. We need more groups to think about and decide on the answers to > questions like the above. > [0] - https://github.com/coreos/fedora-coreos-tracker/issues/225 [1] - https://apps.fedoraproject.org/datagrepper/id?id=2020-81657c3b-9703-431a-8c3c-0b409743fac4&is_raw=true&size=extra-large [2] - https://github.com/coreos/fedora-coreos-tracker/blob/master/Design.md#release-streams > [1]: > https://meetbot-raw.fedoraproject.org/fedora-meeting-1/2020-10-22/f33-final-go_no_go-meeting.2020-10-22-17.00.log.html > [2]: https://github.com/release-engineering/productmd > -- > Adam Williamson > Fedora QA > IRC: adamw | Twitter: adamw_ha > https://www.happyassassin.net > > > _______________________________________________ > devel mailing list -- de...@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org >
_______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
