Seems reasonable to me! On Mon, Oct 30, 2023 at 7:28 AM Wei Lee <[email protected]> wrote:
> +1 > > Best, > Wei > > > On Oct 30, 2023, at 5:36 AM, Utkarsh Sharma < > [email protected]> wrote: > > > > +1 for the implementing the process, have little clue about the > provider. :) > > > > Thanks, > > Utkarsh Sharma > > > > On Sun, Oct 29, 2023 at 10:59 PM Pierre Jeambrun <[email protected]> > wrote: > >> > >> +1 > >> > >> Le ven. 27 oct. 2023 à 19:16, Aritra Basu <[email protected]> a > >> écrit : > >> > >>> Sounds like a good time to set the process up. +1 from me as well. > >>> > >>> -- > >>> Regards, > >>> Aritra Basu > >>> > >>> On Fri, Oct 27, 2023, 6:42 PM Vincent Beck <[email protected]> > wrote: > >>> > >>>> I like that. I also think it is important to have a process to remove > >>>> provider if needed. +1 > >>>> > >>>> On 2023/10/27 09:00:25 Jarek Potiuk wrote: > >>>>>> I think in the case of Qubole it is pretty easy to remove it from > the > >>>>> provider codebase. I'm pretty sure that almost no one even noticed > this > >>>>> removal. > >>>>> > >>>>> Yeah. Agree. This one is pretty "obvious" that's why I would like to > >>>> create > >>>>> a process for doing it along the way so that in the future if we have > >>>> other > >>>>> non-obvious cases we can just "follow the process". > >>>>> > >>>>> BTW.That would be really great to have as with it, we will have a > >>>> complete > >>>>> "life-cycle" of the providers. > >>>>> > >>>>> * we know how we approve the new ones > >>>>> * we know how we release new versions (Elad is amazing to release > them > >>>>> every few weeks) > >>>>> * we know how we maintain back-compatibility (bumping min-version of > >>>>> Airflow that we regularly do ) > >>>>> * we know how to involve stakeholders to system-test them and to make > >>>> them > >>>>> work in a stable way when they connect to external services (Amazon > >>>> works, > >>>>> Google in Progres, Open API and others promised by Astronomer) > >>>>> * we know how to involve stakeholders with mixed-governance in case > >>> they > >>>>> want older releases (never happened yet but we know how to do it) > >>>>> * we know how to suspend and resume them when they prove to be > >>>> problematic > >>>>> and pass resolution of that to external stakeholders (happened with > >>>> Yandex > >>>>> - both suspend and resume) > >>>>> * we (will finally) know how to retire them when we decide we do not > >>> want > >>>>> to maintain them - except security fixes - any more > >>>>> > >>>>> That will pretty much complete our process of "life-cycle" management > >>> for > >>>>> providers. > >>>>> > >>>>> J. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Oct 26, 2023 at 10:00 PM Jarek Potiuk <[email protected]> > >>> wrote: > >>>>> > >>>>>> > >>>>>>> I suggest also removing it from pypi for security reasons. If there > >>>> is a > >>>>>>> security issue with it then the issue will remain with us. > >>>>>>> > >>>>>>> > >>>>>> I am quite sure we still have to handle security issues if someone > >>>> finds > >>>>>> them. releasing such a provider will still be possible using the > >>>> tag/branch > >>>>>> and we will be obliged to release a new version IF it is still used > >>>> and a > >>>>>> security issue is found. > >>>>>> Removal of packages from PyPI does not remove our obligation to fix > a > >>>>>> security problem. We have also source packages released via > >>>>>> downloads.apache.org and archives.apache.org - and those we can't > >>>> remove > >>>>>> either. > >>>>>> > >>>>>> I think this is one of the obligations of the Foundation being the > >>>>>> "steward" of software it releases - that's why there is also the > >>>> attic PMC > >>>>>> to manage projects that PMC is unable to support (projects are moved > >>> to > >>>>>> attic when they fail a roll call from the board with less than 2 PMC > >>>>>> members confirming that they are still there and ready to handle > >>>> releases > >>>>>> if needed. It's also being discussed to be more formal for the CRA > >>>>>> regulations right now - "stewards" of the software put on the market > >>>> should > >>>>>> be responsible for handling security issues in a timely manner). The > >>>> act of > >>>>>> release with 3 +1s of PMC is a legal act of the Foundation placing > >>>> software > >>>>>> on the market and we can't make it "unhappen". > >>>>>> > >>>>>> > >>>>>>> B. > >>>>>>> > >>>>>>> Sent from my iPhone > >>>>>>> > >>>>>>>> On 26 Oct 2023, at 20:20, Jarek Potiuk <[email protected]> wrote: > >>>>>>>> > >>>>>>>> Hello Airflow community, > >>>>>>>> > >>>>>>>> How do we feel about removing the Qubole provider completely > >>>> (leaving > >>>>>>> only > >>>>>>>> old releases in PyPI? > >>>>>>>> > >>>>>>>> On September 1 2023 ( > >>>>>>>> https://lists.apache.org/thread/p394d7w7gc7lz61g7qdthl96bc9kprxh) > >>>> the > >>>>>>>> Qubole operator ws suspended. > >>>>>>>> > >>>>>>>> Due to the reasons described in the thread (Qubole got acquired > >>> and > >>>> the > >>>>>>>> service is generally abandoned) there is pretty much no chance for > >>>> it > >>>>>>> to be > >>>>>>>> resumed. > >>>>>>>> > >>>>>>>> I'd love to remove it completely and introduce a process where we > >>>> can do > >>>>>>>> similar things in the future for other providers if we decide to > >>> do > >>>> so. > >>>>>>>> > >>>>>>>> I checked in the Attic project in the ASF (this is where abandoned > >>>>>>> project > >>>>>>>> of the ASF get moved to) and it seems that just removing part of > >>> the > >>>>>>>> project that has an active PMC is not going through attic > >>>>>>>> https://issues.apache.org/jira/browse/ATTIC-218 . We are free to > >>>>>>> define our > >>>>>>>> rules for that and I would like to use the opportunity to hash it > >>>> out > >>>>>>> and > >>>>>>>> propose a process (similarly to suspension) and criteria to remove > >>>>>>>> providers from being maintained by us. > >>>>>>>> > >>>>>>>> It's more than suspension. We will completely stop updating the > >>>> related > >>>>>>>> code (right now some automated changes can still be applied and > >>>>>>> suspended > >>>>>>>> providers can be resumed with simple PR). I would like to have the > >>>>>>> "next" > >>>>>>>> step after "suspending" - removal. > >>>>>>>> > >>>>>>>> Roughly - we send PROPOSAL followed by VOTE (or immediately VOTE > >>> in > >>>>>>>> obvious cases) with justification, PMC members only have the > >>> binding > >>>>>>> votes > >>>>>>>> (similar as for releases). > >>>>>>>> > >>>>>>>> Only git history will remain - all the rest will be removed > >>>> (including > >>>>>>>> extra) - no traces of the provider remain in the next MINOR > >>> release > >>>>>>> (2.8.0 > >>>>>>>> in the case of Quibole). The provider will still be in PyPI and > >>>>>>> historical > >>>>>>>> releases will be in https://archive.apache.org . If someone would > >>>> like > >>>>>>> to > >>>>>>>> bring back such a provider, It should go through the same process > >>>> as a > >>>>>>> new > >>>>>>>> provider (voting/consensus). And we might reject it. > >>>>>>>> > >>>>>>>> WDYT? Any comments for such an approach / process ? > >>>>>>>> > >>>>>>>> J. > >>>>>>> > >>>>>>> > >>> --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: [email protected] > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > >>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
