BTW. Another example of an exception would be a security fix. We **migh** find a security issue that will require a breaking change. But then again - the breaking change might **only** cover that security fix - we do not have to remove all the deprecations in this case (but we could if we decide so - very much depending on nature of those deprecations)
On Thu, Feb 29, 2024 at 12:36 PM Jarek Potiuk <ja...@potiuk.com> wrote: > I think it's a good idea to codify the rules, indeed what Elad mentioned, > we **mostly** follow a very similar approach, but it's not written > anywhere. > > > Re: points of Elad: > > 1. I am ok with 6 months, but I agree here we should not have "strict" > limits. It's fine to have some default (i.e. accumulate deprecating changes > for some time - max. 6 months) - but I think the policy should have an > option to apply exceptions - for example when we have a major change in > many libraries and we need to make some breaking changes (ads case) we > shall do it (but in this case we don't have to immediately remove all > deprecations - they can wait for the regular (say 6 months) breaking change > that will be coming. The users can - if needed - even in new versions of > Airflow - downgrade to previous provider versions in case they have problem > with it. > > I like the idea of having some time (6 months) where we can very safely > say "OK, enough is enough, lets remove deprecations" - and this proposal > addresses it nicely. Currently it's purely subjective judgment. One small > thing here to clarify for "accumulating" deprecations - I think we should > make sure the deprecation is present in already released minor/patchlevel > version to be able to remove it (so even if it is deprecated 2 months ago > but we had release with the deprecation, it's ok to remove it in the next > breaking change. > > 2. Agree (and again here "strict" is not good I think). I'd treat that 6 > months as a baseline. Before that - we should have very good reason to make > a breaking change (and we might decide to do it partially only) - after - > we are free to remove things that have been already announced as deprecated > and we should consider making breaking change whenever we release a new > version for whatever reason (but again - not mandatory). I's just an > indication that tells us "Last breaking release was 6 months ago, it's now > OK to remove all the deprecations without having an extremely good reason". > > Very much agree with 3. -> this should be the same as for other providers. > We should adopt the same rule for all of them. And if we apply > the adjustments above - we should - I think - be ok to apply it to any > provider I think. > > J > > > > On Thu, Feb 29, 2024 at 12:20 PM Elad Kalif <elad...@apache.org> wrote: > >> This is not too much different than what we already do. >> We deprecate then we remove. We almost never create breaking change >> without >> deprecating first. >> >> I'd like to raise a few notes: >> >> 1. Major releases can happen frequently. I see no problem with it. This is >> more a question of what are the changes. If google has 3 major releases >> one >> with breaking change to ads, another to leveldb and 3rd to cloud these are >> not related one to another and unlikely that there is 1 user that uses all >> of these together. We create major releases when needed and we keep a >> close >> eye and challenge why a change must be a breaking change. If possible we >> will always prefer to be backward compatible. >> 2. Time based deprecations are not a good approach. Some changes require >> much more time to be acted upon and others may be trivial. >> As a rule we almost never cut a breaking change release just to remove >> deprecations. >> 3. Setting a policy per provider is not the right way to go. We have a >> shared governance model for provider handling and I as release manager >> will >> do what I can to accommodate requests from companies who participate in >> managing the provider. Which means that if Google wants to remove >> deprecations and ask for a breaking change release most likely we will >> accommodate their request. My point of view is that the company knows what >> is best for their own users. I don't think that what is right for Google >> must bind AWS or Microsoft. 1 policy goes against the shared >> governance idea. >> >> >> >> On Thu, Feb 29, 2024 at 11:04 AM Eugen Kosteev <eu...@kosteev.com> wrote: >> >> > Hi. >> > >> > I would like to discuss/propose a deprecation policy for >> > apache-airflow-providers-google package, mostly for deprecating Airflow >> > operators (but not limited to it). >> > >> > *Some background:* >> > Airflow google provider package (as most of other provider packages in >> > Airflow) has a tendency to constantly evolve. >> > There are cases when its features or operators/hooks have to be >> deprecated. >> > The typical example of such a case is “refactoring” which leads to the >> > renaming of operators, parameters or public methods. Such refactorings >> > maybe consequence of: >> > - using a new dependent library (that causes some methods to be >> deprecated) >> > - using a new version of API (that causes some parameters to be >> deprecated) >> > - restructuring/migration of the operators (that causes some operators >> to >> > be deprecated) >> > Given the need of a “deprecation” procedure, it looks essential to >> > establish policy around it to have users of Airflow google provider >> package >> > clearly understand when and how to adapt their DAGs. >> > >> > *Preambula (versioning of the package):* >> > As mentioned in “Airflow’s release process and version policy” ( >> > >> > >> https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#airflow-s-release-process-and-version-policy >> > ) >> > google provider package (and others) should follow SemVer, meaning that >> any >> > breaking changes should be released together with bumping major version >> of >> > the package. >> > The change is considered to be a breaking (in the context of google >> > providers package) if a DAG that was working before, stops to work after >> > the change. >> > >> > *My proposal (for discussion):* >> > The entire procedure of deprecating (either method, parameter or >> operator) >> > consists of two steps: >> > - Emission of the deprecation warning message >> > Example of the message: >> > “”” >> > The “given” parameter is deprecated and will be removed after >> dd.mm.yyyy. >> > Please use “this” parameter instead. >> > “”” >> > - Once the date of the deprecated method/parameter/operator is passed, >> it >> > can be removed (together with bumping major version of the package) >> > >> > The format of the warning message may vary, but it has to contain: >> > - What should be used instead >> > - The date after which the method/parameter/operator will be removed >> > >> > *Additional considerations:* >> > - Bumping a major version of the Airflow google provider package >> shouldn’t >> > happen very frequently (presumably not more frequently than 6 months), >> > therefore when it happens it may accumulate removals of something that >> was >> > communicated to be removed even a couple of months ago. (related >> discussion >> > https://github.com/apache/airflow/blob/main/PROVIDERS.rst) >> > Example: if today is 2025-12-20 and we are releasing new major version >> of >> > the package, we can/should remove all deprecated >> > methods/parameters/operators which have date prior to 2025-12-20 - it >> can >> > be November 2025, October 2025 or even earlier, since previous bump of >> > major version happened e.g. in summer of 2025. >> > - By default all deprecations should allow a 6 months time period until >> > they will be removed and not available. This period will give Airflow >> users >> > enough time and flexibility to update their DAGs before actual removal >> > happens. >> > On a case by case basis this period can be adjusted given specific >> > circumstances (e.g. in case deprecation is because of underlying API >> sunset >> > which can happen earlier than in 6 months). >> > >> > Apart from deprecation warning messages that will be printed as logs, >> this >> > information will be put as well in: >> > - Airflow documentation >> > - Release notes of Airflow google provider package >> > >> > Please, let me know what you think about this. >> > >> > -- >> > Eugene >> > >> >
