Hello everyone, We seem to have general consensus about the need and general approach for removing providers. We had discussion about it in https://lists.apache.org/thread/x4gt2h5hql7j04jj0v7v7kzzv1nkrzxy with generally everyone being in favour of both - removing Qubole and capturing the removal process in the "lifecycle" of Provider documentation we already have.
The PR about the updated Process is here https://github.com/apache/airflow/pull/35277 and already got a few approvals and went through a few rounds of comments (thanks to those who commented already). I think it's already in the state that we can consider it "settled" (there might still be English/ typo/small corrections) but I guess nothing substantial, so I am calling for a Lazy Consensus on the content of it. If there are any comments like that - feel free to comment in the PR. The PR contains a bit of restructuring the doc content as well but below I copy the current proposal of the removal process. There is no need to respond to it, if there will be no objections, lazy consensus will be reached on Monday, 6th November midnight CEST - together with consensus of Qubole removal and if consensus is reached I will proceed with the removal. J. ------- Content of "Removing community providers" chapter Removing community providers The providers can be removed from main branch of Airflow when the community agrees that there should be no more updates to the providers done by the community - except maybe potentially security fixes found. There might be various reasons for the providers to be removed: the service they connect to is no longer available the dependencies for the provider are not maintained any more and there is no viable alternative there is another, more popular provider that supersedes community provider etc. etc. Each case of removing provider should be discussed individually and separate [VOTE] thread should start, where regular rules for code modification apply (following the Apache Software Foundation voting rules). In cases where the reasons for removal are obvious, and discussed before, also [LAZY CONSENSUS] thread can be started. Generally speaking a discussion thread [DISCUSS] is advised before such removal and sufficient time should pass (at least a week) to give a chance for community members to express their opinion on the removal. There are the following consequences (or lack of them) of removing the provider: One last release of the provider is done with documentation updated informing that the provider is no longer maintained by the Apache Airflow community - linking to this page. This information should also find its way to the package documentation and consequently - to the description of the package in PyPI. An [ANNOUNCE] thread is sent to the devlist and user list announcing removal of the provider The released provider packages remain available on PyPI and in the Archives of the Apache Software Foundation, while they are removed from the Downloads . Also it remains in the Index of the Apache Airflow Providers documentation at Airflow Documentation with note (not maintained) next to it. The code of the provider is removed from main branch of the Apache Airflow repository - including the tests and documentation. It is no longer built in CI and dependencies of the provider no longer contribute to the CI image/constraints of Apache Airflow for development and future MINOR release. The provider is removed from the list of Apache Airflow extras in the next MINOR Airflow release The dependencies of the provider are removed from the constraints of the Apache Airflow (and the constraints are updated in the next MINOR release of Airflow) In case of confirmed security issues that need fixing that are reported to the provider after it has been removed, there are two options: * in case there is a viable alternative or in case the provider is anyhow not useful to be installed, we might issue advisory to the users to remove the provider (and use alternatives if applicable) in case the users might still need the provider, we still might decide to release new version of the provider with security issue fixed, starting from the source code in Git history where the provider was last released. This however, should only be done in case there are no viable alternatives for the users. Removed provider might be re-instated as maintained provider, but it needs to go through the regular process of accepting new provider described above.
