I am +1 for (1) to (3) [also assuming that (2) is mostly like a doc bugfix!]
For (4) I am hesitant and would rather be conservative. Every
cherry-pick has a risk to break something in old codebase. As branches
change over time and backport PRs are clearly less cautious reviewed it
might lead to introduced inconsistencies which might not be covered in
tests. I would also not back-port just for the sake of easier later
maintenance for other cherry-picks. But for (4) the rule might not be
too strict and every rule is made for exceptions which in (4) might be
likely. But I would not advertise to backport refactorings if no clear
benefit.
On 23.06.25 15:33, Pavankumar Gopidesu wrote:
Thanks Jarek, for starting this discussion,
I agree with all the points.
The real intention behind to backport
https://github.com/apache/airflow/pull/51992 is , this area has a lot of
ongoing development going and I felt it's worth porting to v3-0-test.
I myself faced situations where I tried to backport some doc changes , I
failed because of conflicts that previous changes were not ported back on
the same file etc or conflicts with some lines.
So I am very strong on this if there are any areas with heavy development
activity going. I feel it's worth backporting the changes and IMHO i
don't see any problem.
Pavan
On Mon, Jun 23, 2025 at 2:23 PM Jarek Potiuk <ja...@potiuk.com> wrote:
I think We are approaching it from the same point of view, just that we
have different conclusions.
I agree. I think that simply there is one special case that we should
"allow". Details below.
For 4, I do not have strong opinions on either front, but defining what
to
do and what not should
probably be that if that change makes it easy to backport something else,
maybe have it?
Yep. That's **exactly** my proposal. I do not want to backport **all**
refactorings. That would be stupid. In my definition of 4. I want to
cherry-pick those pre-commit when it's small, and automated and when we can
easily anticipate it will make it quite likely someone (soon) will do
another cherry-pick that will be conflicting.
Yes. It's not "0/1" and quite a bit personal judgment, and yes it **might
not** result in conflict (so we might end up with YAGNI), but I think we
should define (and trust committers judgment) when they do it, because they
want to prevent "others" to have problems.
For example in case of https://github.com/apache/airflow/pull/51992 -> the
only reason why I think it makes sense to cherry-pick it is because it is
in the area of task-sdk that has been recently bug-fixed quite a number of
times and there are open related issues to this area that make it (IMHO)
quite likely to result in conflict. I think there are other areas like that
(UI for example) where we clearly cherry-pick a number of changes, because
there are some pretty active "bug-fixing" in this area as 3.0 had still a
number of low priority but either known or anticipated bugs that are likely
to be fixed in 3.0.3, 3.0.4
So just to clarify - I do not want to cherry-pick "all" refactoring, but
leave to the judgment of the commiter merging the request (and author) to
decide to cherry-pick such change anticipating it will make life easier for
others.
J
On Mon, Jun 23, 2025 at 2:57 PM Amogh Desai <amoghdesai....@gmail.com>
wrote:
Agree with point 1 - 3 definitely.
For 4, I do not have strong opinions on either front, but defining what
to
do and what not should
probably be that if that change makes it easy to backport something else,
maybe have it?
For ex:
*PR 1 changes file1.py and file2.py*
*PR 2 changes some lines in file2.py*
*We backport PR 1 as it's a bug fix and do not for PR 2 as its some
refactoring.*
*Now while trying to backport PR 3 (bugfix), it conflicts and needs PR 2
to
be picked*
*to land PR 3.*
Thanks & Regards,
Amogh Desai
On Mon, Jun 23, 2025 at 5:02 PM Ash Berlin-Taylor <a...@apache.org>
wrote:
I think We are approaching it from the same point of view, just that we
have different conclusions.
Points 1-3 I agree with.
We do already have this written up
https://github.com/apache/airflow/blob/130e9600443e06c08acc1b28c69a62c858d6e6a2/dev/README_RELEASE_AIRFLOW.md?plain=1#L116-L129
On 4 I think of it in terms that “every change carries some risk” — in
the
linked example of 51992 the risk is almost zero, but generally: 3.0.x
are
meant to be bug fix releases on top of 3.0.0, and if it’s not fixing a
bug
we don’t back port it. The one exception I have to this is if the
change
is
needed to make it easy to backport a change that is a bug without
conflicts.
I think our default approach has to be we don’t back port a change
unless
it is fixing a bug, otherwise the risk of “oh I’ll just fix this” ends
up
introducing more bugs than we fix. Stability of a Minor release series
is
my primary desire, and not changing things more than we have to is the
best
way I know of doing that.
Things are slightly different now that we have automated cherry-picks
but
I still don’t think it is worth porting refactoring automatically. It’s
extra change and risk for almost zero benefit to users is my view.
-ash
On 23 Jun 2025, at 11:43, Jarek Potiuk <ja...@potiuk.com> wrote:
BTW. I'd be happy to capture result of this discussion if we can get
to a
consensus or vote eventually in the "cherry-picking" guidelines.
On Mon, Jun 23, 2025 at 12:42 PM Jarek Potiuk <ja...@potiuk.com>
wrote:
I wanted to start a discussion on "things that we cherry-pick" (to
vX_Z
branch).
I think there are different opinions on what kind of changes should
be
cherry-picked and it might be a good idea to agree on a common
approach.
I think (following the comment of Ash here)
https://github.com/apache/airflow/pull/51992#issuecomment-2995632849
that
we can use a very simplistic and (I'd say) dogmatic approach "only
cherry-pick bug fixes. Full stop". But I believe (and past
experience
from
a lot of cherry-picking that I've been doing - multiple times
helping
to
bring past branches to be green and spending countless hours on it,
that it
should be a bit more nuanced.
I would love to see what others think, but from my experience those
are
the things that we **should** cherry-pick:
1) bug-fixes (of course)
2) doc changes (when they are improvements or filling gaps)
3) dev tool changes (every time we did not, it resulted in hours of
my
time when things were breaking and I tried to reconcile it)
4) results of automated refactorings that have very low risks (in
the
areas that are likely to have cherry-picks)
t) - is non-controversial I think
2) - is also relatively non-controversial and very low risk and
gives
our
users a chance to get better docs earlier (even today for example I
cherry
picked this one: https://github.com/apache/airflow/pull/52068 -
because
one of my friends who tries to learn Airflow 3 pinged me that
"ConfiuguringRuff" link that we have in 3.0.2 leads to 404 NOT found
3) - it had always bitten us if we stopped cherry-picking dev tool
changes. The thing is that external dependencies change all the time
and we
are continuously catching up with those, also we improve, speed up
and
simplify the tooling - and often things that worked when branch was
cut,
does not work today - countless, countless hours lost in one or two
branches when we stopped doing it - I think even once or twice I had
to
just copy over most (but not all) the code from main to the branch
and
commit one single "catch-up dev tooling with main" big change
4) Is likely most controversial - example here:
https://github.com/apache/airflow/pull/51992/ - those are the kind
of
(really small) changes that are done in "active" area (i.e. area
that
had
and will have a lot of cherry-picks anyway, but they are done with
automated refactoring - like renaming variables and such. This
introduces
clarity and readability, so this is good we are doing them. But if
we
do
not cherry-pick them and then we cherry-pick any change that touches
the
same code, this lead to a conflict. Conflicts are frustrating,
especially
those kinds - you never know what you should do - should you "merge"
this
naming change with your change? or should you leave the original
namiing,
or should you try to find the past commit that changed it and
cherry-pick
it as well? This paired with the fact that we are using
cherry-picker
that
allows to cherry-pick stuff very quickly, automatically and
painlessly
when
there are no conflicts, make me think that yes - we should cherry
-pick
those changes proactively as a service to those contributors who
will
follow up with their cherry-picking. It's just "good service" and
helping
others who will come after you.
That's how my definition of "what we should cherry-pick" is...
I wonder what others think about it ?
J.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
