I think once we go through the current bugfisinv process, we should come-back to the discussion.
There are a couple of preparatory things in general that I think can start happen regardless (there are a couple of unresolved questions in our provider process - such as dependent providers discussion), we need to make a few changes to the release process to be able to handle parallel "batches" of providers, and we should "properly" produce source packages for the batches versioned with date of release ( https://github.com/apache/airflow/issues/47343 issue describing it). I will attempt to make those things that are needed for the discussions to continue - to be addressed before - so I might start a separate discussion or two - would love those interested to take part in those discussions in the meantime. J. On Wed, Sep 17, 2025 at 4:00 PM Vincent Beck <[email protected]> wrote: > Aside from a few details that still need refinement, I support this idea. > I believe it’s the only viable way to continue expanding the number of > integrations Airflow supports without overloading the provider release > manager. The responsibility for releasing providers would be scaled out, > rather than concentrated as it is today, which I see as a great > improvement. That said, I agree with Jarek’s comments—responsibilities > across individuals will need to be clearly defined, agreed upon, and well > understood to ensure the Airflow ecosystem continues to function smoothly. > > On 2025/09/17 12:39:45 Stephen Mallette wrote: > > As a more casual observer of what happens here in Airflow, I don't have > any > > first hand knowledge of the problems faced beyond what was described in > the > > Motivation section of the proposal. That said, I did read through it and > > was curious about the deprecation path that was described. I understand > > that the path leaves open room for exception and that the quantitative > > definitions are likely there to help guide PMC decisions, but I wonder if > > it would make sense to describe wider exceptions for more mature > providers > > that don't require lots of new features/maintenance. That might help > reduce > > the scope of the PMC quarterly provider review while also reducing the > > chance of providers gaming the system and/or sparing them a bit of worry > > that they miss a quarterly chance to discuss an exception with the PMC. > > > > > > > > On Tue, Sep 16, 2025 at 9:14 PM Jarek Potiuk <[email protected]> wrote: > > > > > Comment from my side: I think this is a really good idea to discuss > > > **what** we need from the community side to make it "easy" for > providers to > > > be contributed and maintained - without over-burdening the maintainers. > > > This is by far the most important aspect of this proposal - and I > think it > > > would be fantastic if we hear feedback from Amazon, Google. Astronomer, > > > Teradata and others who recently submitted providers to Airflow - and > of > > > course all maintainers that might be worried about impact it will have > on > > > our maintenance efforts. > > > > > > I think before we approve this one and start following it - all parties > > > involved (maintainers, release managers ,contributors, PMC members, > > > stakeholders) will need to agree on understanding: > > > > > > * what is required from them (everyone - stakeholders, contributors. > > > maintainers, PMC members, release managers will have their parts in the > > > process). > > > * what they will have to commit to > > > * what kind of tooling will be needed to make it as efficient as > possible > > > without overburdening everyone > > > > > > We are way ahead with tooling and modularisation of airflow than we > were a > > > year ago and I think changes and improvements in the tooling and > process > > > updates needed to get the process in place are within our reach now. > This > > > is the reason why I think it's the **right** time to discuss it. It > will > > > take a couple of months to complete all the needed changes, but it all > > > seems doable. > > > > > > Also - I have discussed it with Apache Software Foundation leadership > at > > > Community Over Code in Minneapolis - and they are interested and > supported > > > in us trying this approach where we want to get Stakeholders more > involved > > > and more accountable, while keeping the Apache Way approach, so we > should > > > have green light from the Foundation on it. > > > > > > J. > > > > > > > > > On Tue, Sep 16, 2025 at 12:02 PM Vikram Koka via dev < > > > [email protected]> > > > wrote: > > > > > > > Dear Airflowers, > > > > > > > > Every few days, we get a request for adding a new Provider (aka > > > > Integration) into Apache Airflow. We would like to say yes to most > if not > > > > all of these, but we have had challenges in being able to absorb many > > > more > > > > providers into Airflow. > > > > > > > > We have had multiple conversations around this in the past, and we > would > > > > like to discuss a proposal to enable us to adopt many more > integrations > > > to > > > > Airflow. At this stage, this is an early draft intended to get your > > > > feedback. > > > > > > > > I know a lot of us are working on releasing Airflow 3.1.0, so I don't > > > > expect a lot of conversation this week, but wanted to socialize this > with > > > > the broader community at the same time. > > > > > > > > Proposal > > > > < > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Provider+lifecycle+update+proposal > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Provider+lifecycle+update+proposal > > > > > > > > Best regards, > > > > Vikram, Jarek, Kaxil, Pavan, and Ash > > > > -- > > > > > > > > Vikram Koka > > > > Chief Strategy Officer > > > > Email: [email protected] > > > > > > > > > > > > <https://www.astronomer.io/> > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
