On Fri, Mar 3, 2023 at 3:42 AM Daniel P. Berrangé <berra...@redhat.com>
wrote:

> A few weeks back there was a little snafu (since resolved) with the
> capstone package in EPEL.
>
> This package is being introduced to RHEL 9.2, and thus has been added
> to CentOS 9 Stream repos. A bug was filed against capstone in EPEL
> to notify that capstone should be retired when RHEL 9.2 is released:
>
>   https://bugzilla.redhat.com/show_bug.cgi?id=2124181
>
> Unfortunately the package was accidentally retired immediately rather
> than waiting until 9.2 release. This is an easy mistake for a maintainer
> to make when they see a bug whose title is "Remove capstone from epel9",
> so I don't want anyone to blame the maintainer for this accident.
>
> Yes the comments make it clear that removal should wait until 9.2
> release date, but I can totally understand how actions will be guided
> by the initial bug subject if you're just skimming. It was pointed out
> that this risk has been identified before:
>
>
> https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/thread/LUWUIN5BRPJZGVB5PIH625PFV6IBD6YO/
>
> and now we have a real world example of it going wrong exactly as
> was predicted.
>
>
> My bigger concern though is what happens when RHEL-9.2 is actually
> released for real. The bug remains open and targets (immediate)
> retirement of capstone from EPEL on / near 9.2 release date.
>
>
> This doesn't feel like it takes into account the real world usage of
> enterprise distributions. The kind of people/organizations who deploy
> RHEL are rarely going to rush into upgrading their deployment processes
> to use 9.2 immediately. They may have all sorts of review and testing
> processes to go through, integration of 3rd party apps and services,
> validating against hardware, before they consider starting deployment
> of RHEL 9.2.
>
> All this time they will continue deploying machines with 9.1, despite
> 9.2 being available, so won't get access to new 9.2 packages.
>
> If we immediately retire capstone when 9.2 is released, aren't we
> going to break any ability to keep deploying capstone from EPEL on
> existing/new 9.1 installs ?
>
> There is also the case of the RHEL rebuilds whose users consume EPEL
> packages. Depending how quick they are, the rebuild distros might not
> have their 9.2 rebuild ready for some days/weeks/months after RHEL-9.2
> is first available. My projects' upstream CI is all based on AlmaLinux
> and I don't want to see it broken again by premature capstone retirement
> from EPEL.
>
> Unless I'm mistaken in some way, it feels like we should *NOT* sync
> the removal of packages from EPEL with their release in RHEL 9.2. It
> needs to be synced with user deployment of 9.2, which might be somewhat
> later.
>
>
>
> The package added to RHEL should (must) have a newer NEVR than the
> existing package in EPEL, so even if we leave it in EPEL, it will
> be ignored if the newer version is present in RHEL repos.
>
> The main important thing I see is that the EPEL branch in dist-git
> must be made read-only and/or prevented from submitting new builds,
> once the package has been built in CentOS Stream. ie to ensure the
> EPEL NEVR doesn't creep back above the pending RHEL package's NEVR.
>
>
> If that's done it should be fine to leave it in EPEL for a prolonged
> period after 9.2 is released, to allow sufficient time for users to
> actually switch from deploying 9.1 over to 9.2.
>
> I guess someone might say that EPEL only targets the very latest
> y-stream of RHEL, so continuing to use 9.1 after 9.2 is released
> is unsupported ?  Even if that is technically the case, I think
> in practice users won't take that into account, and thus expect
> it to keep working and generally that should be fine. Only if a
> EPEL package is rebuilt such that it depends on a newly added
> API/ABI is that a problem.
>
> With regards,
> Daniel
>

Hi Daniel,
 Thank you for your insight as an end user.  Sometimes repo and package
maintainers are always trying to get to the latest and greatest and rush
things.

We are currently in the middle of changing the workflow of retiring these
packages.  We are changing it so that the package maintainer doesn't do the
removing.  It will be automated, or semi-automated, so there is a
consistent time when all of them are removed.

The first step is re-working the bug that get's opened.
It will still be against the package that is to be retired, and linked to
the tracker.
The subject and initial comments will tell the maintainer that they do not
need to do anything, that it will be taken care of for them.
An automation of some type (currently it is manual) will add another
comment assuring them that it will be done automatically and there is
nothing for them to do.
The automation will then assign the bug to the appropriate person/group.

But then comes the part that you talk about, and it's a good thing to have
a discussion about.
When do you actually remove the packages from the EPEL repository?
It has been agreed that it will be after both Alma and Rocky have their
latest release out.
But how long after?
Immediately after?  a month? 6 months?

I personally am leaning towards a month after.
Here is my reasoning.
At the time a new RHEL release is released, we take a snapshot of the EPEL
repo and put it in the archives.
So all the packages that were built, and run on RHEL 9.1 are available in
that archive.
There are always some packages in EPEL that no longer install on the newer
libraries of a new release.  Depending on the library, that might be a
couple, it might be alot.
It takes roughly a month for those EPEL packages to get built, go through
testing, and released.
Anyway, if some users are still doing new installs of RHEL 9.1 (or
compatible) after that month, then they should probably add the epel 9.1
archive to their yum repositories.

There is the argument that why remove them at all if they are a lower NVR
than RHEL's?
But then you will have to figure out which packages are higher than the
RHEL versions.
It's much easier and cleaner to remove them.

Troy
_______________________________________________
epel-devel mailing list -- epel-devel@lists.fedoraproject.org
To unsubscribe send an email to epel-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Reply via email to