https://github.com/apache/airflow/pull/52148 -> Proposed PR. Let's iterate there if there are comments.
On Tue, Jun 24, 2025 at 11:25 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Yeah. I think what I am really after is expressed by Wei: +0.5 vote - > i.e. we should be fine with **some** selected cherry-picks that are not > bug-fixes that we proactively cherry-pick and do not "complain" that this > is wrong because technically it's not a bugfix. I think It would be great > if in every such cherry-pick we rationalise why we are doing it: > > "I know it's not a bugfix, but because this is an actively developed part > that is cherry-picked heavily, let me cherry-pick that one to make it > easier for others to cherry-pick bugfixes". > > If we are all on-board with that (seems so) then I can update the > cherry-pick description to add it as possible exception. > > J. > > On Tue, Jun 24, 2025 at 10:14 AM Kaxil Naik <kaxiln...@gmail.com> wrote: > >> It looks like everyone is mostly on the same page based on all the emails >> - >> so no comments :) . >> >> On Tue, 24 Jun 2025 at 07:45, Wei Lee <weilee...@gmail.com> wrote: >> >> > I’m +1 for 1-3 (assuming the doc changes relate to the backported >> version). >> > +0.5 for 4. I hope that changes not related to new features will be >> > backported when feasible; however, we can skip them if the required >> effort >> > is substantial. This is because failing to backport these items could >> > potentially lead to future conflicts with points 1 or 3. >> > >> > Best, >> > Wei >> > >> > > On Jun 24, 2025, at 5:32 AM, Jens Scheffler >> <j_scheff...@gmx.de.INVALID> >> > wrote: >> > > >> > > I am +1 for (1) to (3) [also assuming that (2) is mostly like a doc >> > bugfix!] >> > > >> > > For (4) I am hesitant and would rather be conservative. Every >> > cherry-pick has a risk to break something in old codebase. As branches >> > change over time and backport PRs are clearly less cautious reviewed it >> > might lead to introduced inconsistencies which might not be covered in >> > tests. I would also not back-port just for the sake of easier later >> > maintenance for other cherry-picks. But for (4) the rule might not be >> too >> > strict and every rule is made for exceptions which in (4) might be >> likely. >> > But I would not advertise to backport refactorings if no clear benefit. >> > > >> > > On 23.06.25 15:33, Pavankumar Gopidesu wrote: >> > >> Thanks Jarek, for starting this discussion, >> > >> >> > >> I agree with all the points. >> > >> >> > >> The real intention behind to backport >> > >> https://github.com/apache/airflow/pull/51992 is , this area has a >> lot >> > of >> > >> ongoing development going and I felt it's worth porting to v3-0-test. >> > >> I myself faced situations where I tried to backport some doc changes >> , I >> > >> failed because of conflicts that previous changes were not ported >> back >> > on >> > >> the same file etc or conflicts with some lines. >> > >> So I am very strong on this if there are any areas with heavy >> > development >> > >> activity going. I feel it's worth backporting the changes and IMHO i >> > >> don't see any problem. >> > >> >> > >> Pavan >> > >> >> > >> >> > >> On Mon, Jun 23, 2025 at 2:23 PM Jarek Potiuk <ja...@potiuk.com> >> wrote: >> > >> >> > >>>> I think We are approaching it from the same point of view, just >> that >> > we >> > >>> have different conclusions. >> > >>> >> > >>> I agree. I think that simply there is one special case that we >> should >> > >>> "allow". Details below. >> > >>> >> > >>>> For 4, I do not have strong opinions on either front, but defining >> > what >> > >>> to >> > >>>> do and what not should >> > >>>> probably be that if that change makes it easy to backport something >> > else, >> > >>>> maybe have it? >> > >>> Yep. That's **exactly** my proposal. I do not want to backport >> **all** >> > >>> refactorings. That would be stupid. In my definition of 4. I want to >> > >>> cherry-pick those pre-commit when it's small, and automated and when >> > we can >> > >>> easily anticipate it will make it quite likely someone (soon) will >> do >> > >>> another cherry-pick that will be conflicting. >> > >>> >> > >>> Yes. It's not "0/1" and quite a bit personal judgment, and yes it >> > **might >> > >>> not** result in conflict (so we might end up with YAGNI), but I >> think >> > we >> > >>> should define (and trust committers judgment) when they do it, >> because >> > they >> > >>> want to prevent "others" to have problems. >> > >>> >> > >>> For example in case of https://github.com/apache/airflow/pull/51992 >> > -> the >> > >>> only reason why I think it makes sense to cherry-pick it is because >> it >> > is >> > >>> in the area of task-sdk that has been recently bug-fixed quite a >> > number of >> > >>> times and there are open related issues to this area that make it >> > (IMHO) >> > >>> quite likely to result in conflict. I think there are other areas >> like >> > that >> > >>> (UI for example) where we clearly cherry-pick a number of changes, >> > because >> > >>> there are some pretty active "bug-fixing" in this area as 3.0 had >> > still a >> > >>> number of low priority but either known or anticipated bugs that are >> > likely >> > >>> to be fixed in 3.0.3, 3.0.4 >> > >>> >> > >>> So just to clarify - I do not want to cherry-pick "all" refactoring, >> > but >> > >>> leave to the judgment of the commiter merging the request (and >> author) >> > to >> > >>> decide to cherry-pick such change anticipating it will make life >> > easier for >> > >>> others. >> > >>> >> > >>> J >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> On Mon, Jun 23, 2025 at 2:57 PM Amogh Desai < >> amoghdesai....@gmail.com> >> > >>> wrote: >> > >>> >> > >>>> Agree with point 1 - 3 definitely. >> > >>>> >> > >>>> For 4, I do not have strong opinions on either front, but defining >> > what >> > >>> to >> > >>>> do and what not should >> > >>>> probably be that if that change makes it easy to backport something >> > else, >> > >>>> maybe have it? >> > >>>> >> > >>>> For ex: >> > >>>> *PR 1 changes file1.py and file2.py* >> > >>>> *PR 2 changes some lines in file2.py* >> > >>>> >> > >>>> *We backport PR 1 as it's a bug fix and do not for PR 2 as its some >> > >>>> refactoring.* >> > >>>> >> > >>>> *Now while trying to backport PR 3 (bugfix), it conflicts and needs >> > PR 2 >> > >>> to >> > >>>> be picked* >> > >>>> *to land PR 3.* >> > >>>> >> > >>>> >> > >>>> Thanks & Regards, >> > >>>> Amogh Desai >> > >>>> >> > >>>> >> > >>>> On Mon, Jun 23, 2025 at 5:02 PM Ash Berlin-Taylor <a...@apache.org> >> > >>> wrote: >> > >>>>> I think We are approaching it from the same point of view, just >> that >> > we >> > >>>>> have different conclusions. >> > >>>>> >> > >>>>> Points 1-3 I agree with. >> > >>>>> >> > >>>>> We do already have this written up >> > >>>>> >> > >>> >> > >> https://github.com/apache/airflow/blob/130e9600443e06c08acc1b28c69a62c858d6e6a2/dev/README_RELEASE_AIRFLOW.md?plain=1#L116-L129 >> > >>>>> On 4 I think of it in terms that “every change carries some risk” >> — >> > in >> > >>>> the >> > >>>>> linked example of 51992 the risk is almost zero, but generally: >> 3.0.x >> > >>> are >> > >>>>> meant to be bug fix releases on top of 3.0.0, and if it’s not >> fixing >> > a >> > >>>> bug >> > >>>>> we don’t back port it. The one exception I have to this is if the >> > >>> change >> > >>>> is >> > >>>>> needed to make it easy to backport a change that is a bug without >> > >>>> conflicts. >> > >>>>> >> > >>>>> I think our default approach has to be we don’t back port a change >> > >>> unless >> > >>>>> it is fixing a bug, otherwise the risk of “oh I’ll just fix this” >> > ends >> > >>> up >> > >>>>> introducing more bugs than we fix. Stability of a Minor release >> > series >> > >>> is >> > >>>>> my primary desire, and not changing things more than we have to is >> > the >> > >>>> best >> > >>>>> way I know of doing that. >> > >>>>> >> > >>>>> Things are slightly different now that we have automated >> cherry-picks >> > >>> but >> > >>>>> I still don’t think it is worth porting refactoring automatically. >> > It’s >> > >>>>> extra change and risk for almost zero benefit to users is my view. >> > >>>>> >> > >>>>> -ash >> > >>>>> >> > >>>>>> On 23 Jun 2025, at 11:43, Jarek Potiuk <ja...@potiuk.com> wrote: >> > >>>>>> >> > >>>>>> BTW. I'd be happy to capture result of this discussion if we can >> get >> > >>>> to a >> > >>>>>> consensus or vote eventually in the "cherry-picking" guidelines. >> > >>>>>> >> > >>>>>> On Mon, Jun 23, 2025 at 12:42 PM Jarek Potiuk <ja...@potiuk.com> >> > >>>> wrote: >> > >>>>>>> I wanted to start a discussion on "things that we cherry-pick" >> (to >> > >>>> vX_Z >> > >>>>>>> branch). >> > >>>>>>> >> > >>>>>>> I think there are different opinions on what kind of changes >> should >> > >>> be >> > >>>>>>> cherry-picked and it might be a good idea to agree on a common >> > >>>> approach. >> > >>>>>>> I think (following the comment of Ash here) >> > >>>>>>> >> > >>> >> https://github.com/apache/airflow/pull/51992#issuecomment-2995632849 >> > >>>>> that >> > >>>>>>> we can use a very simplistic and (I'd say) dogmatic approach >> "only >> > >>>>>>> cherry-pick bug fixes. Full stop". But I believe (and past >> > >>> experience >> > >>>>> from >> > >>>>>>> a lot of cherry-picking that I've been doing - multiple times >> > >>> helping >> > >>>> to >> > >>>>>>> bring past branches to be green and spending countless hours on >> it, >> > >>>>> that it >> > >>>>>>> should be a bit more nuanced. >> > >>>>>>> >> > >>>>>>> I would love to see what others think, but from my experience >> those >> > >>>> are >> > >>>>>>> the things that we **should** cherry-pick: >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> 1) bug-fixes (of course) >> > >>>>>>> 2) doc changes (when they are improvements or filling gaps) >> > >>>>>>> 3) dev tool changes (every time we did not, it resulted in >> hours of >> > >>> my >> > >>>>>>> time when things were breaking and I tried to reconcile it) >> > >>>>>>> 4) results of automated refactorings that have very low risks >> (in >> > >>> the >> > >>>>>>> areas that are likely to have cherry-picks) >> > >>>>>>> >> > >>>>>>> t) - is non-controversial I think >> > >>>>>>> >> > >>>>>>> 2) - is also relatively non-controversial and very low risk and >> > >>> gives >> > >>>>> our >> > >>>>>>> users a chance to get better docs earlier (even today for >> example I >> > >>>>> cherry >> > >>>>>>> picked this one: https://github.com/apache/airflow/pull/52068 - >> > >>>> because >> > >>>>>>> one of my friends who tries to learn Airflow 3 pinged me that >> > >>>>>>> "ConfiuguringRuff" link that we have in 3.0.2 leads to 404 NOT >> > found >> > >>>>>>> >> > >>>>>>> 3) - it had always bitten us if we stopped cherry-picking dev >> tool >> > >>>>>>> changes. The thing is that external dependencies change all the >> > time >> > >>>>> and we >> > >>>>>>> are continuously catching up with those, also we improve, speed >> up >> > >>> and >> > >>>>>>> simplify the tooling - and often things that worked when branch >> was >> > >>>> cut, >> > >>>>>>> does not work today - countless, countless hours lost in one or >> two >> > >>>>>>> branches when we stopped doing it - I think even once or twice I >> > had >> > >>>> to >> > >>>>>>> just copy over most (but not all) the code from main to the >> branch >> > >>> and >> > >>>>>>> commit one single "catch-up dev tooling with main" big change >> > >>>>>>> >> > >>>>>>> 4) Is likely most controversial - example here: >> > >>>>>>> https://github.com/apache/airflow/pull/51992/ - those are the >> kind >> > >>> of >> > >>>>>>> (really small) changes that are done in "active" area (i.e. area >> > >>> that >> > >>>>> had >> > >>>>>>> and will have a lot of cherry-picks anyway, but they are done >> with >> > >>>>>>> automated refactoring - like renaming variables and such. This >> > >>>>> introduces >> > >>>>>>> clarity and readability, so this is good we are doing them. But >> if >> > >>> we >> > >>>> do >> > >>>>>>> not cherry-pick them and then we cherry-pick any change that >> > touches >> > >>>> the >> > >>>>>>> same code, this lead to a conflict. Conflicts are frustrating, >> > >>>>> especially >> > >>>>>>> those kinds - you never know what you should do - should you >> > "merge" >> > >>>>> this >> > >>>>>>> naming change with your change? or should you leave the original >> > >>>>> namiing, >> > >>>>>>> or should you try to find the past commit that changed it and >> > >>>>> cherry-pick >> > >>>>>>> it as well? This paired with the fact that we are using >> > >>> cherry-picker >> > >>>>> that >> > >>>>>>> allows to cherry-pick stuff very quickly, automatically and >> > >>> painlessly >> > >>>>> when >> > >>>>>>> there are no conflicts, make me think that yes - we should >> cherry >> > >>>> -pick >> > >>>>>>> those changes proactively as a service to those contributors who >> > >>> will >> > >>>>>>> follow up with their cherry-picking. It's just "good service" >> and >> > >>>>> helping >> > >>>>>>> others who will come after you. >> > >>>>>>> >> > >>>>>>> That's how my definition of "what we should cherry-pick" is... >> > >>>>>>> >> > >>>>>>> I wonder what others think about it ? >> > >>>>>>> >> > >>>>>>> J. >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>> >> > >>>>> >> --------------------------------------------------------------------- >> > >>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> > >>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >> > >>>>> >> > >>>>> >> > > >> > > --------------------------------------------------------------------- >> > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> > > For additional commands, e-mail: dev-h...@airflow.apache.org >> > > >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> > For additional commands, e-mail: dev-h...@airflow.apache.org >> > >> > >> >
