On 28/04/2023 09:32, Frode Nordahl wrote:
On Mon, Apr 17, 2023 at 9:46 PM Han Zhou <hz...@ovn.org> wrote:

On Mon, Apr 17, 2023 at 12:01 PM Ilya Maximets <i.maxim...@ovn.org> wrote:

On 4/17/23 19:20, Han Zhou wrote:


On Mon, Apr 17, 2023 at 9:24 AM Numan Siddique <num...@ovn.org <mailto:
num...@ovn.org>> wrote:

On Mon, Apr 17, 2023 at 5:57 AM Dumitru Ceara <dce...@redhat.com
<mailto:dce...@redhat.com>> wrote:

On 4/17/23 11:55, Dumitru Ceara wrote:
On 4/14/23 18:01, Han Zhou wrote:


On Fri, Apr 14, 2023 at 2:16 AM Dumitru Ceara <dce...@redhat.com
<mailto:dce...@redhat.com>
<mailto:dce...@redhat.com <mailto:dce...@redhat.com>>> wrote:

On 4/14/23 04:15, Han Zhou wrote:
On Thu, Apr 13, 2023 at 12:44 PM Han Zhou <hz...@ovn.org
<mailto:hz...@ovn.org>
<mailto:hz...@ovn.org <mailto:hz...@ovn.org>>> wrote:

I backported this series to the last release branch-23.03 and
the LTS
branch-22.03.
The code base has changed a lot so was mostly manually
backported
instead
of cherry-pick.


Thanks for that but I think we shouldn't skip stable branches in
between.  We likely still have users on those that won't upgrade
to the
latest 23.03 but instead to a new z-stream release (when that
happens)
of the stable branch.

I cherry picked your backports and made sure the tests pass and
then
pushed them to branches 22.12, 22.09 and 22.06.

Thanks Dumitru for helping backporting. I was uncertain about
whether we
should continue backporting without skipping branches, as our
growing
number of released branches makes this approach increasingly
difficult
to sustain.

The release process document
(https://docs.ovn.org/en/latest/internals/release-process.html <
https://docs.ovn.org/en/latest/internals/release-process.html>
<https://docs.ovn.org/en/latest/internals/release-process.html <
https://docs.ovn.org/en/latest/internals/release-process.html>>) doesn't
specify how long we should maintain non-LTS branches. My
understanding
was that we should primarily maintain LTS and the latest releases
for
regular bug fixes, while critical/security fixes would be applied
to all
branches. It appears we have different interpretations of this
process.
While it's beneficial to backport to all branches, I thought this
was
more of a convenience when there weren't too many branches to
manage. In
the worst case, a patch may require manual backporting to each
branch if
new features make the codebase in each branch unique. I also
question
what differentiates an LTS branch if we're backporting to all
branches
anyway. My assumption was that users opting for a non-LTS branch
should
be prepared to upgrade to newer releases, given the "short-term"
maintenance of these branches. Is my understanding incorrect?

Hi Han,

I changed the subject of the email to better reflect the new
discussion.

There's this statement in the release-process document: "LTS
releases
will receive bug fixes until the point that another LTS is
released. At
that point, the old LTS will receive an additional year of
critical and
security fixes."

Which further differentiates an LTS release from a non-LTS release.

If you are saying "the old LTS will receive an additional year of
critical and security fixes" is the different part, I agree. But still, in
this case, for critical and security fixes, when applied to the LTS for the
*additional* year, do you agree that we will skip non-LTS releases? If not
skipping, then it makes no difference.


Perhaps we should discuss and formalize our backporting practice
going
forward, as the number of branches will only continue to grow in
the
coming months and years. cc @Mark Michelson
<mailto:mmich...@redhat.com <mailto:mmich...@redhat.com>>  @Numan
Siddique <mailto:num...@ovn.org <mailto:num...@ovn.org>>


Checking the OVS/OVN release-process documents again it's indeed
not
explicitly specified that non-LTS stable release branches are
supposed
to get all bug fixes.  But the OVS document has this addition:

"While branches other than LTS and the latest release are not
formally
maintained, the OVS project usually provides stable releases for
these
branches for at least 2 years, i.e. stable releases are provided
for the
  last 4 release branches.  However, these branches may not include
all
the fixes that LTS has in case backporting is not straightforward
and
developers are not willing to spend their time on that (this mostly
affects branches that are older than the LTS, because backporting
to LTS
implies backporting to all intermediate branches)."

This last part really makes me think that we SHOULD backport to all
intermediate branches: "...  because backporting to LTS
implies backporting to all intermediate branches".


I agree with this.

If we consistently backport to all intermediate releases, the LTS
release appears to function more as a reference point in the series of
releases, rather than a version with additional maintenance support. This
is ok if we all agree to it.

Maybe we should make this rule (or some other backport strategy we
decide to use from this point on) explicit indeed.

First of all, the backporting process is already documented in OVN:

https://docs.ovn.org/en/latest/internals/contributing/backporting-patches.html

"The maintainer first applies the patch to master, then backports the
  patch to each older affected tree, as far back as it goes or at least
  to all currently supported branches. This is usually each branch back
  to the most recent LTS release branch."

Thanks for the pointer, and sorry for missing it earlier. Probably we
should combine these two documents for OVN because both mentioned some part
of backport processes, and remove the irrelevant part (like
userspace/kernel related discussion for OVS).


No-one updated the documentation in general since migration from master
to main, but that's not the point.

Backports are done for each intermediate branch to avoid regressions
on upgrades, which may be very bad for users and not a great thing to
have in general (also makes it harder to bisect in which branch the
issue was introduced).

This should remain this way, I think, unless OVN decides that intermediate
branches are not supported at all and never get any backports.  i.e. the
release schema similar to DPDK, where all xx.11 releases are supported for
a few years, but intermediate releases are fully abandoned the moment the
next one is out.


"the LTS release appears to function more as a reference point in the
  series of releases, rather than a version with additional maintenance
  support"

The reason why it appears this way is a bit orthogonal.  The point
of extra maintenance support is preparing of official minor releases.
Minor releases on LTS and the latest stable should be much more
frequent than on other branches that are not fully supported.
For example, I'm trying to release a new minor version of OVS LTS and
the latest stable every ~2 months, while other "unsupported" branches
may receive minor releases once in 4-6 months.

The problem with OVN project that makes us ask these questions is that
OVN doesn't really prepare any minor releases any regularly at all,
while it should.  OVN 22.03 LTS received 2 stables releases since release
in March 2022 after people asking for them.  In comparison, OVS 2.17 LTS
received 6 new minor versions within approximately same time frame.

So, the original point of a long term support was to keep backporting
fixes all the way down and prepare minor releases more frequently on
this one particular branch.   And backporting all the way down, according
to the current documentation, means backporting to all intermediate
branches.

Thanks for the explanation. Now I see the point of a LTS release. Releasing
minor versions more frequently does make it more useful.
So according to the current documented process we should not skip branches
even for the "additional" year of a LTS, although the DPDK approach seems
more attractive to me considering the amount of release branches between
two LTS in OVN.

fwiw; the avoidance of regression on upgrade is very important for the
distributions. Since DPDK is brought up as an example, a consequence
of their backport strategy is that distributions do not put any
intermediate releases into stable releases.

So if OVN were to adopt a similar strategy the consequence would be
that only the LTS releases get wide spread exposure, meaning less
field exposure to new features and longer time before end users get
their hands on them.

This boils down to a question of how to balance resources, and it is
up to you to decide, just wanted to make sure the decision is made
with all information available.


Very good point, +1.

It might also be worth noting some reasons why DPDK does it like that so you can compare and contrast with OVN.

DPDK typically sees up to 100 fixes per LTS per month. Usually 50%+ of them are to drivers. So effort for maintenance (especially for older LTSs) and multi-vendor release validation can be quite a bit. This effort is a limiting factor for how many branches can be reasonably maintained and have validated releases.

You'd need to consider the maturity of the project too. Probably for many just having fast IO on their NIC of choice is enough and they can stick on a DPDK LTS for a few years. The DPDK model might not suit if a project is getting 'must have' new features per release because it is earlier in it's maturity.

HTH,
Kevin.

_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to