+1 Best, Wei
> On Oct 30, 2023, at 5:36 AM, Utkarsh Sharma > <utkarsh.sha...@astronomer.io.INVALID> wrote: > > +1 for the implementing the process, have little clue about the provider. :) > > Thanks, > Utkarsh Sharma > > On Sun, Oct 29, 2023 at 10:59 PM Pierre Jeambrun <pierrejb...@gmail.com> > wrote: >> >> +1 >> >> Le ven. 27 oct. 2023 à 19:16, Aritra Basu <aritrabasu1...@gmail.com> a >> écrit : >> >>> Sounds like a good time to set the process up. +1 from me as well. >>> >>> -- >>> Regards, >>> Aritra Basu >>> >>> On Fri, Oct 27, 2023, 6:42 PM Vincent Beck <vincb...@apache.org> wrote: >>> >>>> I like that. I also think it is important to have a process to remove >>>> provider if needed. +1 >>>> >>>> On 2023/10/27 09:00:25 Jarek Potiuk wrote: >>>>>> I think in the case of Qubole it is pretty easy to remove it from the >>>>> provider codebase. I'm pretty sure that almost no one even noticed this >>>>> removal. >>>>> >>>>> Yeah. Agree. This one is pretty "obvious" that's why I would like to >>>> create >>>>> a process for doing it along the way so that in the future if we have >>>> other >>>>> non-obvious cases we can just "follow the process". >>>>> >>>>> BTW.That would be really great to have as with it, we will have a >>>> complete >>>>> "life-cycle" of the providers. >>>>> >>>>> * we know how we approve the new ones >>>>> * we know how we release new versions (Elad is amazing to release them >>>>> every few weeks) >>>>> * we know how we maintain back-compatibility (bumping min-version of >>>>> Airflow that we regularly do ) >>>>> * we know how to involve stakeholders to system-test them and to make >>>> them >>>>> work in a stable way when they connect to external services (Amazon >>>> works, >>>>> Google in Progres, Open API and others promised by Astronomer) >>>>> * we know how to involve stakeholders with mixed-governance in case >>> they >>>>> want older releases (never happened yet but we know how to do it) >>>>> * we know how to suspend and resume them when they prove to be >>>> problematic >>>>> and pass resolution of that to external stakeholders (happened with >>>> Yandex >>>>> - both suspend and resume) >>>>> * we (will finally) know how to retire them when we decide we do not >>> want >>>>> to maintain them - except security fixes - any more >>>>> >>>>> That will pretty much complete our process of "life-cycle" management >>> for >>>>> providers. >>>>> >>>>> J. >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Oct 26, 2023 at 10:00 PM Jarek Potiuk <ja...@potiuk.com> >>> wrote: >>>>> >>>>>> >>>>>>> I suggest also removing it from pypi for security reasons. If there >>>> is a >>>>>>> security issue with it then the issue will remain with us. >>>>>>> >>>>>>> >>>>>> I am quite sure we still have to handle security issues if someone >>>> finds >>>>>> them. releasing such a provider will still be possible using the >>>> tag/branch >>>>>> and we will be obliged to release a new version IF it is still used >>>> and a >>>>>> security issue is found. >>>>>> Removal of packages from PyPI does not remove our obligation to fix a >>>>>> security problem. We have also source packages released via >>>>>> downloads.apache.org and archives.apache.org - and those we can't >>>> remove >>>>>> either. >>>>>> >>>>>> I think this is one of the obligations of the Foundation being the >>>>>> "steward" of software it releases - that's why there is also the >>>> attic PMC >>>>>> to manage projects that PMC is unable to support (projects are moved >>> to >>>>>> attic when they fail a roll call from the board with less than 2 PMC >>>>>> members confirming that they are still there and ready to handle >>>> releases >>>>>> if needed. It's also being discussed to be more formal for the CRA >>>>>> regulations right now - "stewards" of the software put on the market >>>> should >>>>>> be responsible for handling security issues in a timely manner). The >>>> act of >>>>>> release with 3 +1s of PMC is a legal act of the Foundation placing >>>> software >>>>>> on the market and we can't make it "unhappen". >>>>>> >>>>>> >>>>>>> B. >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>>> On 26 Oct 2023, at 20:20, Jarek Potiuk <ja...@potiuk.com> wrote: >>>>>>>> >>>>>>>> Hello Airflow community, >>>>>>>> >>>>>>>> How do we feel about removing the Qubole provider completely >>>> (leaving >>>>>>> only >>>>>>>> old releases in PyPI? >>>>>>>> >>>>>>>> On September 1 2023 ( >>>>>>>> https://lists.apache.org/thread/p394d7w7gc7lz61g7qdthl96bc9kprxh) >>>> the >>>>>>>> Qubole operator ws suspended. >>>>>>>> >>>>>>>> Due to the reasons described in the thread (Qubole got acquired >>> and >>>> the >>>>>>>> service is generally abandoned) there is pretty much no chance for >>>> it >>>>>>> to be >>>>>>>> resumed. >>>>>>>> >>>>>>>> I'd love to remove it completely and introduce a process where we >>>> can do >>>>>>>> similar things in the future for other providers if we decide to >>> do >>>> so. >>>>>>>> >>>>>>>> I checked in the Attic project in the ASF (this is where abandoned >>>>>>> project >>>>>>>> of the ASF get moved to) and it seems that just removing part of >>> the >>>>>>>> project that has an active PMC is not going through attic >>>>>>>> https://issues.apache.org/jira/browse/ATTIC-218 . We are free to >>>>>>> define our >>>>>>>> rules for that and I would like to use the opportunity to hash it >>>> out >>>>>>> and >>>>>>>> propose a process (similarly to suspension) and criteria to remove >>>>>>>> providers from being maintained by us. >>>>>>>> >>>>>>>> It's more than suspension. We will completely stop updating the >>>> related >>>>>>>> code (right now some automated changes can still be applied and >>>>>>> suspended >>>>>>>> providers can be resumed with simple PR). I would like to have the >>>>>>> "next" >>>>>>>> step after "suspending" - removal. >>>>>>>> >>>>>>>> Roughly - we send PROPOSAL followed by VOTE (or immediately VOTE >>> in >>>>>>>> obvious cases) with justification, PMC members only have the >>> binding >>>>>>> votes >>>>>>>> (similar as for releases). >>>>>>>> >>>>>>>> Only git history will remain - all the rest will be removed >>>> (including >>>>>>>> extra) - no traces of the provider remain in the next MINOR >>> release >>>>>>> (2.8.0 >>>>>>>> in the case of Quibole). The provider will still be in PyPI and >>>>>>> historical >>>>>>>> releases will be in https://archive.apache.org . If someone would >>>> like >>>>>>> to >>>>>>>> bring back such a provider, It should go through the same process >>>> as a >>>>>>> new >>>>>>>> provider (voting/consensus). And we might reject it. >>>>>>>> >>>>>>>> WDYT? Any comments for such an approach / process ? >>>>>>>> >>>>>>>> J. >>>>>>> >>>>>>> >>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>> >>>>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>> >>>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
