----- Original Message -----
> From: "Stephen Gallagher" <sgall...@redhat.com>
> To: devel-annou...@lists.fedoraproject.org, "Development discussions related
> to Fedora"
> <devel@lists.fedoraproject.org>
> Sent: Tuesday, March 31, 2020 5:31:38 PM
> Subject: Fedora 33 System-Wide Change Proposal: ELN Buildroot and Compose V3
>
> I sent out the V2 version of the Change on Friday and then promptly
> managed to injure myself and be away from email until today. I've read
> through the email threads again this morning and I decided that,
> rather than try to address them one by one, I'd try again with a V3
> that hopefully answers some of the repeated questions and concerns on
> that list.
>
> Please see the newly-updated
> https://fedoraproject.org/wiki/Changes/ELN_Buildroot_and_Compose
> for more details[1].
>
>
> [1]
> https://fedoraproject.org/w/index.php?title=Changes%2FELN_Buildroot_and_Compose&type=revision&diff=569904&oldid=569809
> _______________________________________________
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
Ok I took the time to read all the previous threads and the proposal as well as
the clarifications.
In general I am in favor of the idea, however I concur with my fellow
packagers, that the current proposal is having some serious issues, and I'm
very reluctant in agreeing with the given rationale of various details.
I also feel that I am iterating a lot of the feedback that was given already
however I believe it was neither heard nor taken into account.
So to start:
> ELN (Enterprise Linux Next) is going to be that place. What we want to do is
> establish a set of RPM conditionals that can be used for the set of packages
> that may need to be built differently for a RHEL-like target as compared to
> a Fedora one. (For example, Fedora often ships with experimental or
> less-commonly-used features enabled for packages that Red Hat would not want
> to support for Enterprise Linux customers).
Some packagers are fine with adding conditionals in their SPECs and that is ok.
However as one of the maintainers of the main python interpreter as well as
various alternative ones, and hundreds of libraries, not only throughout Fedora
but across the RHEL ecosystem, I can say that this change vastly complicates my
work. As a team we maintain approximately 35+ different branches of various
python interpreters and if it was making sense to have single SPECs by using
conditionals, we would pretty much do it without hassle. Different
projects/products have differents needs and different goals (and different
branches but more on that later).
I could just add RHEL and SCL conditionals however not only it would clutter
the SPEC file to insane amounts of unnecessary craft, the only people who would
know how to work with it, would be me and my team. I want to encourage
contributions from the community at my packages, not make them more complex and
deter others from contributing.
So in that respect, in Fedora I wouldn't personally add any RHEL (or ELN)
conditionals, as our RHEL work can vary significantly. And it would drive
potential collaborators away if every change visible in Fedora, would also have
an effect on a black box that would be ELN. Why would they even bother if they
don't have access to that? And what's their motivation to even work with those
conditionals? Am I supposed, for example, to reject PR's that could break ELN
due to its different compiler flags, but have no effect on Fedora, from a user
and a contributor working to better a package on our OS?
Also contributors *are* users of Fedora and driving them away, may very well
drive a portion of our userbase away as well.
"""
As a sister effort to this, the OSCI team will also be implementing automated
tests that will run against ELN composes and provide non-blocking feedback to
the package maintainers about potential issues in the Enterprise Linux
configuration.
"""
I don't like this part either. Feedback is still feedback blocking or not.
Seeing a big red sign that a change broke ELN is just nagging people to fix
issues that might not even affect them in any significant way. I would like the
ELN testing feedback to be opt-in for packagers who do not maintain something
in ELN or RHEL and only the actual RHEL maintainer to get the notification. If
not, that is unnecessary noise.
"""
The reasoning behind this approach is so that we break as little as possible
when we implement this new build and compose. Existing packages that are using
conditionals such as if 0%{?rhel} > 7 will transition over to building "like
RHEL" automatically. Any packages that do not have a shared Fedora/RHEL
specfile will continue to build as normal. There will be some unknown number
of packages whose existing conditionals will need to be updated to account for
ELN. At present, we are estimating that there will be around 200 packages that
need such attention. The ELN SIG will be providing guidance and pull-requests
to assist with this.
"""
The number of packages here is very optimistic and frankly quite unreal. Having
worked on mass scale package changes, from Python rebases, to decoupling
python2 from RHEL8 I can assure you this number would much much higher, in my
opinion at least. Guidance and pull requests, while helpful, shouldn't be
required for packagers who don't want them, whatever their reasons might be.
"""
The %{eln} variable will be set primarily to allow for the possibility of
needing conditionals that apply to ELN only and should not carry over to RHEL,
but we expect its use to be exceptionally rare.
"""
I don't understand this part. So the %{eln} macro will be in Fedora and ELN,
but not actually RHEL?
"
Such fixes will be done via a pull request that states a time limit. We want
the regular maintainers to see / comment / commit, but we also don't want
things to stall for months and get forgotten about.
For less trivial fixes, we may provide a pull-request or use other methods to
communicate with the maintainers to determine a path forwards that is
acceptable to them.
"""
"""
As long as your package builds in ELN then just maintain your package like
normal. If there is a build failure, the ELN SIG may provide a PR as described
above or will discuss alternative approaches on an individual basis with you.
The ELN SIG will not dictate how you must proceed, but will try to find a
solution that you are comfortable with.
"""
It is reasonable to provide a pull request to fix potential issues with ELN.
What is not reasonable, is to impose a time limit and also expect the
maintainers to follow up with that. And what exactly would be the way forward
for packagers that do not want those conditionals in their package, while it
also fails to build in ELN, potentially blocking other packages? Would you
merge it as a proven packager, despite possible drawbacks? I can easily see a
game of revertions if that was to happen. So please explain what other possible
approaches there would be in that case, and I hope it is not to actually
convince the packager to accept the conditionals.
"""
This adds no value to the current approach where Red Hat maintainers would
manually merge their changes into the internal build infrastructure. There's
no way to automate the sync from the master branch to the eln branch that
wouldn't break and require maintainer involvement. Attempting to branch only
individual packages would introduce significant complexity in the build
process as well, leading to far more opportunity for bugs. Lastly, even the
most diligent of maintainers can forget to sync every change to a new branch,
thus leaving us in a situation where the eln branch has fallen behind and is
no longer providing an accurate view of whether the package is still building
or functioning in that environment.
"""
Automatically synced branches or symbolic references to branches is a great
solution to that, and I really like the idea that Fabio proposed to provide an
optional eln branch for packagers who either do not want to merge the changes,
or if the change is too complex to make sense for Fedora.
"""
Classically, Fedora sees a flurry of activity from Red Hat developers in the
run-up to a new Red Hat Enterprise Linux release. These Red Hat teams
frantically attempt to push a few years' downstream effort up into Fedora
before it is branched for the next RHEL development. It is not uncommon for
these teams to disappear downstream again once this has happened. This harms
both Fedora and Red Hat with the lack of upstream involvement. With ELN, we
hope to make Fedora Rawhide a more appealing (and canonical) place for those
developers to work. This should increase the amount of ongoing developer
involvement in Fedora, and also make Fedora a more valuable resource for Red
Hat, which can help to ensure funding and support.
"""
That is true to an extend. I don't see that as a failure with the procedures
though, more like a failure to communicate to those teams/developers, to stress
just how important Fedora is for our downstream work. And honestly I haven't
interacted with many developers who don't think Fedora is important enough, so
that point from my experience is a bit moot, in respect to pushing such an
intrusive change. I'm obviously biased here, but looking, as an example at the
state of the Python, Ruby, Perl ecosystem across Fedora and RHEL, you can see
the Fedora work is not only important, but the main drive for a healthy RHEL
ecosystem. And I'm happy to say my management understands that.
"""
ELN artifacts will be made available for testing and development purposes, but
we will not be shipping any content produced from ELN directly to the general
public (such as on the standard mirror network or via getfedora.org). This does
not necessarily mean that they will be shipped directly from Koji, merely that
they won't come from the standard locations.
"""
It would be better to provide artifacts, there is no reason to keep this in a
black box, maybe I'd like to spin VM's, maybe containers. Just updating a
regular rawhide installation to eln in order to test a simple fix is not
something I would do very enthusiastically.
Some more thoughts:
Since the current Fedora infrastructure is already stretched thin, especially
with modularity, koschei rebuilds, the CI and so on, has a possible impact on
the load been considered/measured? I know for starters that the tasks will have
a lower priority, however has any assessment been made to get a rough idea on
what and if any additional resources will be required?
I'd also like to point out that all the past system-wide changes, except from
maybe rebases of important packages, have always had detailed descriptions on
how they were going to be implemented and a well laid-out contingency plan.
They weren't mere ideas, and while I understand that not all questions can be
answered at this point, starting with a very high level goal and iterating on
it, is not something to be done as a fedora system-wide change, not at least
since some details need to be cemented on the aforementioned points.
A system-wide change and the fedora devel thread of it, is a place to gather
feedback from the community, incorporate that and move forward from there, not
somewhere where blind trust should be placed on the change owners and accuse
people and even elected representatives of ignoring feedback. It really does
seem like projection when placing the whole conversation into context.
I understand that especially during those trying times, everyone might be a bit
on edge while trying to stay productive and within the deadlines, however it
really does seem that the change owners and the community are talking entirely
past each other at this point.
Some time ago, during the discussion of the default modular streams, there was
a big pushback on some aspects of it from the community and the feedback was
rejected and/or ignored in a similar manner that happens throughout those ELN
threads. In the end, the eclipse module did exactly that and it affected a big
portion of our userbase: https://bugzilla.redhat.com/show_bug.cgi?id=1780827
So overall, I'm -1 on the proposal and as things currently stand, I won't be
adding any of those conditionals at the Python SPEC files or any other smaller
library that I maintain. I'll happily reconsider if the feedback is heard and
the pain points have been addressed or are at least planned to be addressed.
P.S. It took me some days to actually draft this mail, as in general those
times, emotions can run wild and I wanted to be objective, if that's even
possible. And I understand that some things might have changed as the thread
progressed, so apologies if any of my points are not valid anymore.
--
Regards,
Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
