Thanks Jarek, for starting this discussion, I agree with all the points.
The real intention behind to backport https://github.com/apache/airflow/pull/51992 is , this area has a lot of ongoing development going and I felt it's worth porting to v3-0-test. I myself faced situations where I tried to backport some doc changes , I failed because of conflicts that previous changes were not ported back on the same file etc or conflicts with some lines. So I am very strong on this if there are any areas with heavy development activity going. I feel it's worth backporting the changes and IMHO i don't see any problem. Pavan On Mon, Jun 23, 2025 at 2:23 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > I think We are approaching it from the same point of view, just that we > have different conclusions. > > I agree. I think that simply there is one special case that we should > "allow". Details below. > > > For 4, I do not have strong opinions on either front, but defining what > to > > do and what not should > > probably be that if that change makes it easy to backport something else, > > maybe have it? > > Yep. That's **exactly** my proposal. I do not want to backport **all** > refactorings. That would be stupid. In my definition of 4. I want to > cherry-pick those pre-commit when it's small, and automated and when we can > easily anticipate it will make it quite likely someone (soon) will do > another cherry-pick that will be conflicting. > > Yes. It's not "0/1" and quite a bit personal judgment, and yes it **might > not** result in conflict (so we might end up with YAGNI), but I think we > should define (and trust committers judgment) when they do it, because they > want to prevent "others" to have problems. > > For example in case of https://github.com/apache/airflow/pull/51992 -> the > only reason why I think it makes sense to cherry-pick it is because it is > in the area of task-sdk that has been recently bug-fixed quite a number of > times and there are open related issues to this area that make it (IMHO) > quite likely to result in conflict. I think there are other areas like that > (UI for example) where we clearly cherry-pick a number of changes, because > there are some pretty active "bug-fixing" in this area as 3.0 had still a > number of low priority but either known or anticipated bugs that are likely > to be fixed in 3.0.3, 3.0.4 > > So just to clarify - I do not want to cherry-pick "all" refactoring, but > leave to the judgment of the commiter merging the request (and author) to > decide to cherry-pick such change anticipating it will make life easier for > others. > > J > > > > > On Mon, Jun 23, 2025 at 2:57 PM Amogh Desai <amoghdesai....@gmail.com> > wrote: > > > Agree with point 1 - 3 definitely. > > > > For 4, I do not have strong opinions on either front, but defining what > to > > do and what not should > > probably be that if that change makes it easy to backport something else, > > maybe have it? > > > > For ex: > > *PR 1 changes file1.py and file2.py* > > *PR 2 changes some lines in file2.py* > > > > *We backport PR 1 as it's a bug fix and do not for PR 2 as its some > > refactoring.* > > > > *Now while trying to backport PR 3 (bugfix), it conflicts and needs PR 2 > to > > be picked* > > *to land PR 3.* > > > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Mon, Jun 23, 2025 at 5:02 PM Ash Berlin-Taylor <a...@apache.org> > wrote: > > > > > I think We are approaching it from the same point of view, just that we > > > have different conclusions. > > > > > > Points 1-3 I agree with. > > > > > > We do already have this written up > > > > > > https://github.com/apache/airflow/blob/130e9600443e06c08acc1b28c69a62c858d6e6a2/dev/README_RELEASE_AIRFLOW.md?plain=1#L116-L129 > > > > > > On 4 I think of it in terms that “every change carries some risk” — in > > the > > > linked example of 51992 the risk is almost zero, but generally: 3.0.x > are > > > meant to be bug fix releases on top of 3.0.0, and if it’s not fixing a > > bug > > > we don’t back port it. The one exception I have to this is if the > change > > is > > > needed to make it easy to backport a change that is a bug without > > conflicts. > > > > > > > > > I think our default approach has to be we don’t back port a change > unless > > > it is fixing a bug, otherwise the risk of “oh I’ll just fix this” ends > up > > > introducing more bugs than we fix. Stability of a Minor release series > is > > > my primary desire, and not changing things more than we have to is the > > best > > > way I know of doing that. > > > > > > Things are slightly different now that we have automated cherry-picks > but > > > I still don’t think it is worth porting refactoring automatically. It’s > > > extra change and risk for almost zero benefit to users is my view. > > > > > > -ash > > > > > > > On 23 Jun 2025, at 11:43, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > > > BTW. I'd be happy to capture result of this discussion if we can get > > to a > > > > consensus or vote eventually in the "cherry-picking" guidelines. > > > > > > > > On Mon, Jun 23, 2025 at 12:42 PM Jarek Potiuk <ja...@potiuk.com> > > wrote: > > > > > > > >> I wanted to start a discussion on "things that we cherry-pick" (to > > vX_Z > > > >> branch). > > > >> > > > >> I think there are different opinions on what kind of changes should > be > > > >> cherry-picked and it might be a good idea to agree on a common > > approach. > > > >> > > > >> I think (following the comment of Ash here) > > > >> > https://github.com/apache/airflow/pull/51992#issuecomment-2995632849 > > > that > > > >> we can use a very simplistic and (I'd say) dogmatic approach "only > > > >> cherry-pick bug fixes. Full stop". But I believe (and past > experience > > > from > > > >> a lot of cherry-picking that I've been doing - multiple times > helping > > to > > > >> bring past branches to be green and spending countless hours on it, > > > that it > > > >> should be a bit more nuanced. > > > >> > > > >> I would love to see what others think, but from my experience those > > are > > > >> the things that we **should** cherry-pick: > > > >> > > > >> > > > >> 1) bug-fixes (of course) > > > >> 2) doc changes (when they are improvements or filling gaps) > > > >> 3) dev tool changes (every time we did not, it resulted in hours of > my > > > >> time when things were breaking and I tried to reconcile it) > > > >> 4) results of automated refactorings that have very low risks (in > the > > > >> areas that are likely to have cherry-picks) > > > >> > > > >> t) - is non-controversial I think > > > >> > > > >> 2) - is also relatively non-controversial and very low risk and > gives > > > our > > > >> users a chance to get better docs earlier (even today for example I > > > cherry > > > >> picked this one: https://github.com/apache/airflow/pull/52068 - > > because > > > >> one of my friends who tries to learn Airflow 3 pinged me that > > > >> "ConfiuguringRuff" link that we have in 3.0.2 leads to 404 NOT found > > > >> > > > >> 3) - it had always bitten us if we stopped cherry-picking dev tool > > > >> changes. The thing is that external dependencies change all the time > > > and we > > > >> are continuously catching up with those, also we improve, speed up > and > > > >> simplify the tooling - and often things that worked when branch was > > cut, > > > >> does not work today - countless, countless hours lost in one or two > > > >> branches when we stopped doing it - I think even once or twice I had > > to > > > >> just copy over most (but not all) the code from main to the branch > and > > > >> commit one single "catch-up dev tooling with main" big change > > > >> > > > >> 4) Is likely most controversial - example here: > > > >> https://github.com/apache/airflow/pull/51992/ - those are the kind > of > > > >> (really small) changes that are done in "active" area (i.e. area > that > > > had > > > >> and will have a lot of cherry-picks anyway, but they are done with > > > >> automated refactoring - like renaming variables and such. This > > > introduces > > > >> clarity and readability, so this is good we are doing them. But if > we > > do > > > >> not cherry-pick them and then we cherry-pick any change that touches > > the > > > >> same code, this lead to a conflict. Conflicts are frustrating, > > > especially > > > >> those kinds - you never know what you should do - should you "merge" > > > this > > > >> naming change with your change? or should you leave the original > > > namiing, > > > >> or should you try to find the past commit that changed it and > > > cherry-pick > > > >> it as well? This paired with the fact that we are using > cherry-picker > > > that > > > >> allows to cherry-pick stuff very quickly, automatically and > painlessly > > > when > > > >> there are no conflicts, make me think that yes - we should cherry > > -pick > > > >> those changes proactively as a service to those contributors who > will > > > >> follow up with their cherry-picking. It's just "good service" and > > > helping > > > >> others who will come after you. > > > >> > > > >> That's how my definition of "what we should cherry-pick" is... > > > >> > > > >> I wonder what others think about it ? > > > >> > > > >> J. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
