> I think in the case of Qubole it is pretty easy to remove it from the provider codebase. I'm pretty sure that almost no one even noticed this removal.
Yeah. Agree. This one is pretty "obvious" that's why I would like to create a process for doing it along the way so that in the future if we have other non-obvious cases we can just "follow the process". BTW.That would be really great to have as with it, we will have a complete "life-cycle" of the providers. * we know how we approve the new ones * we know how we release new versions (Elad is amazing to release them every few weeks) * we know how we maintain back-compatibility (bumping min-version of Airflow that we regularly do ) * we know how to involve stakeholders to system-test them and to make them work in a stable way when they connect to external services (Amazon works, Google in Progres, Open API and others promised by Astronomer) * we know how to involve stakeholders with mixed-governance in case they want older releases (never happened yet but we know how to do it) * we know how to suspend and resume them when they prove to be problematic and pass resolution of that to external stakeholders (happened with Yandex - both suspend and resume) * we (will finally) know how to retire them when we decide we do not want to maintain them - except security fixes - any more That will pretty much complete our process of "life-cycle" management for providers. J. On Thu, Oct 26, 2023 at 10:00 PM Jarek Potiuk <ja...@potiuk.com> wrote: > >> I suggest also removing it from pypi for security reasons. If there is a >> security issue with it then the issue will remain with us. >> >> > I am quite sure we still have to handle security issues if someone finds > them. releasing such a provider will still be possible using the tag/branch > and we will be obliged to release a new version IF it is still used and a > security issue is found. > Removal of packages from PyPI does not remove our obligation to fix a > security problem. We have also source packages released via > downloads.apache.org and archives.apache.org - and those we can't remove > either. > > I think this is one of the obligations of the Foundation being the > "steward" of software it releases - that's why there is also the attic PMC > to manage projects that PMC is unable to support (projects are moved to > attic when they fail a roll call from the board with less than 2 PMC > members confirming that they are still there and ready to handle releases > if needed. It's also being discussed to be more formal for the CRA > regulations right now - "stewards" of the software put on the market should > be responsible for handling security issues in a timely manner). The act of > release with 3 +1s of PMC is a legal act of the Foundation placing software > on the market and we can't make it "unhappen". > > >> B. >> >> Sent from my iPhone >> >> > On 26 Oct 2023, at 20:20, Jarek Potiuk <ja...@potiuk.com> wrote: >> > >> > Hello Airflow community, >> > >> > How do we feel about removing the Qubole provider completely (leaving >> only >> > old releases in PyPI? >> > >> > On September 1 2023 ( >> > https://lists.apache.org/thread/p394d7w7gc7lz61g7qdthl96bc9kprxh) the >> > Qubole operator ws suspended. >> > >> > Due to the reasons described in the thread (Qubole got acquired and the >> > service is generally abandoned) there is pretty much no chance for it >> to be >> > resumed. >> > >> > I'd love to remove it completely and introduce a process where we can do >> > similar things in the future for other providers if we decide to do so. >> > >> > I checked in the Attic project in the ASF (this is where abandoned >> project >> > of the ASF get moved to) and it seems that just removing part of the >> > project that has an active PMC is not going through attic >> > https://issues.apache.org/jira/browse/ATTIC-218 . We are free to >> define our >> > rules for that and I would like to use the opportunity to hash it out >> and >> > propose a process (similarly to suspension) and criteria to remove >> > providers from being maintained by us. >> > >> > It's more than suspension. We will completely stop updating the related >> > code (right now some automated changes can still be applied and >> suspended >> > providers can be resumed with simple PR). I would like to have the >> "next" >> > step after "suspending" - removal. >> > >> > Roughly - we send PROPOSAL followed by VOTE (or immediately VOTE in >> > obvious cases) with justification, PMC members only have the binding >> votes >> > (similar as for releases). >> > >> > Only git history will remain - all the rest will be removed (including >> > extra) - no traces of the provider remain in the next MINOR release >> (2.8.0 >> > in the case of Quibole). The provider will still be in PyPI and >> historical >> > releases will be in https://archive.apache.org . If someone would like >> to >> > bring back such a provider, It should go through the same process as a >> new >> > provider (voting/consensus). And we might reject it. >> > >> > WDYT? Any comments for such an approach / process ? >> > >> > J. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>
