[ https://issues.apache.org/jira/browse/ATTIC-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780044#comment-17780044 ]
Jarek Potiuk commented on ATTIC-218: ------------------------------------ Thanks > Moving one of many packages to attic (or just removing it from the code)? > ------------------------------------------------------------------------- > > Key: ATTIC-218 > URL: https://issues.apache.org/jira/browse/ATTIC-218 > Project: Attic > Issue Type: Task > Reporter: Jarek Potiuk > Priority: Major > > Hello, > We have an interesting situation in Apache Airflow that I would like to have > an advice on. > When we want to remove some small (but with separate release pacakge) part of > the PMC project, should we (somehow?) move the code we remove to attic, or is > it just fine to remove it from sources? > A bit of context: > Apache Airflow is a huge and rather successful PMC (>2600 contributors, tens > of thousands of users) but it also rather complex one. Since Airflow is an > orchestrator and interacts with multiple eservices, instead of a single or > few package we have 1 core, 1 python client and ~ 90 provider packages - each > provider is a separately released convenience package, separately released > source package. Every two weeks or so we release and vote on between few to > all 90 provider packages (there is a well established process of bulk voting. > Example one here: > [https://lists.apache.org/thread/j4b5851g9lqg0jm8yb68f83vqfxzqy14] > We released all providers that are "active" - because we bumped the minimum > version of Airflow according to our policies. > All those providers are managed together with Airflow in a single monorepo > [https://github.com/apache/airflow/tree/main/airflow/providers] > We currently have also well established process of suspending providers that > are - for some reason holding us back (mainly due to dependencies) - we stop > releasing them, we stop updating the code and the code in the repo gets > efffectively ignored by tests/CI/release process. The process, voting, > resuming is well established and being followed (we already suspended - and > later resumed yandex, and currently we suspended qubole provider). This is > really nice for temporary suspensions as it does not hold the development, > but it makes it really easy (just a PR) to fix and resume such provider > (worked very nicely in case of yandex who got resumed after upstream > libraries fixed some dependency issues) > https://github.com/apache/airflow/blob/main/airflow/providers/SUSPENDING_AND_RESUMING_PROVIDERS.rst > However we eventually would like to remove qubole. It's pretty useless now - > because the service (qubole) that it connected to had been acquired and > discontinued > The website is still there but see the note here: > [https://github.com/qubole/qds-sdk-py#where-are-the-maintainers-] (note > added 7 months ago) - there is basically no way any of their libraries will > get updated, and the service does not work any more. Maybe someone would > like to use the code we have there but for us it is a burden (dependencies, > refactors etc. etc.) and we would very much like to just remove it. from > `main` and simply mark the "apache-airflow-providers-qubole" package as > "gone". > So we would like to remove the code and somehow let the users know that the > package we released in the past is "discontinued" in PyPI > [https://pypi.org/project/apache-airflow-providers-qubole/] > So I have a few questions: > 1) should attic PMC be involved when we remove the code or are we fine to > **just** remove it. > 2) any advice on communication / informing our users? > 3) should we do anything about the package we release in PyPI? > Would love to hear what you think. > -- This message was sent by Atlassian Jira (v8.20.10#820010)