I really like this proposal. I feel like it's something we needed for a long time. More comments inline!
On Mon, Jan 13, 2020 at 5:45 PM Aleksandra Fedorova <afedor...@redhat.com> wrote: > Hi, all, > > This topic goes along the lines of Matthew’s Operating System Factory > discussion[1], but with a slightly different angle. It also is the > generalization of the Change we have proposed recently [2] > > So let me start with the problem: > > In Fedora now we have a well known and established process of how to > deal with updates that bring new content through packages. There are > ways to package new content side by side with the old one, to perform > a non-disruptive testing, iterate and upgrade. > > What I think is missing is the way to safely iterate on the process, > which happens after the content is packaged. Examples are: changes to > buildroot configuration as in [2], changes in comps files, for example > required for Minimization Objective [3], changes in compose building > process, changes in Mass Rebuild and so on. > Yes! I felt that a few times in the past. We know how to make our output evolve, how to introduce new RPM versions and make other output-related changes. We test all updates thanks to Bodhi, try and test bigger changes in side tags, etc. But we don't really have a way to safely experiment with what makes it all happen — the Fedora build system. And by "build system", I mean it in the broadest sense, including Koji, compose, configurations, and many other parts). And that's exactly where things like improving packager experience in an actual significant way (that requires some testing and iterations), or performance tuning that affects compatibility — requiring all those technical changes you've listed — happen. Or would happen! > For such changes we have to use an all-or-nothing approach, which > means we either implement the change too early or postpone it for too > long. > ... or don't have space to experiment with bigger changes, potentially limiting our ability to innovate. And when we try to, there is a high risk of negatively impacting our contributors, because, well, production is where the testing happens. > To add flexibility to the Fedora process, I’d like to propose the > concept of alternative buildroots. > Alternative buildroots are a good start. > > ------ > Note: The naming is hard here, and while I tend to call it > “buildroot”, it actually needs to be called “alternative everything, > except the srpms”. > Oh! Ok, that's exactly what I was hoping for and that's even a better start! (Or, you know, all we need :D) > > I think we shouldn’t use the word “variant”, “spin”, “flavour” or > something like that to describe it, as it is strictly the shared > development playground linked to the latest Fedora Rawhide state and > focused on the build and compose process. It is definitely not the > alternative Fedora. It shouldn’t have releases on its own, it has no > branches in the source code and it doesn’t target end-users. > ------ > > The good thing is that, with the distributed CI approach we have > implemented, we are ready to consume the feedback from such > alternative setups in a non-blocking but informative way. I think we > can extend our use of CI machinery so that it simplifies the > development process for everyone, removing redundant heads-ups but > increasing the targeted collaboration in places where it is actually > needed. > > ### Use cases ### > > The first example of such a buildroot is provided by the CPU baseline > testing [1]. > > There will be more, since we may want to work on comps files for > Minimization Objective. > As part of the objective, we avoided such changes as they're quite disruptive. I was actually thinking about trying those "somewhere" on the side, but this proposal sounds like enabling this, done properly as part of our infra, for everyone. Cool! > > There is also one particularly important case of the RHEL bootstrap: > RHEL tries hard to inherit as much as possible from Fedora, but since > both Fedora and RHEL have historically different build and compose > configurations, it is often problematic to deal with dependency and > build issues which arise from it. To the point that RHEL sometimes > just can’t use Fedora as an origin, and needs to find its way around. > > The alternative buildroot targeting CentOS and RHEL would provide an > open shared development environment. There we can work on converging > the RHEL process to Fedora, and also on improving Fedora process with > things, which RHEL and CentOS have discovered. > > ### Proposal ### > > This idea doesn’t quite fit into the Fedora Changes framework, as > there is no actual change to Fedora. And we don't even have a concept > of an Infrastructure Change at this point. Infra Changes are an interesting concept for this. One thing that struck my mind was... Testing alternative approaches on the side without affecting people is not really a change. That said, Changes we do for content, implemented more or less in the main space, force us get there faster. Sure, bigger changes are properly tested in side tags for example, but some are done directly (while being properly announced of course). What I'm trying to say is... I think there is a balance of speed vs. having a safe space to innovate on bigger things. And we definitely need both for infra. > But there are several items one can think about: > > 1) Though as I said above the “buildroot” term is slightly misleading > but it is a start. And the change [2] is a first step, the prototype, > which goes into that direction. Its main focus is the compiler > parameters rather than anything else. > I think this is a good start that will touch many places we'll need to tweak to make them flexible enough to handle multiple build configs and, of course, multiple outputs of those. > 2) The second part of the proposal is to setup a CentOS/RHEL-targeting > buildroot and compose, with a focus on comps, dependency chains and > compose differences. Which I think overlaps in a certain way with the > Minimization Objective. > Will this be a general thing, that will allow others to build such additional pipelines for testing? Or will this be solely for the CentOS/RHEL-targeting part? > > 3) And the third part is to generally develop the alternative > buildroot as a concept. It should be our toolbox item, something > similar to the alternative architectures approach. So that it can be > requested by anyone based on the existence of a SIG or working group, > which would own and maintain it. > I think this answer my question — we'll test it on that one thing, learn some lessons, and based on that we'll come up with a generic way for everyone. And that actually sounds much better than trying to come up with a generic thing first. Did I get that right? ---- I'm actually really excited about this effort. Definitely interested in helping with some of that! > > [1] > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/ZBU4MYRMMAE2Z7DL4NPPECTMX2FBQAVL/ > [2] > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/IFBHS2WKKPKJH6H54OX4DV3U7A4XYOPU/ > [3] > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/IFBHS2WKKPKJH6H54OX4DV3U7A4XYOPU/ > > -- > Aleksandra Fedorova > bookwar > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --------------------------- Senior Software Engineer Red Hat
_______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
