Oh yes. They (or at least one maintainer and looks like quite a few reacting people) are empathising with us and have no objections for removal so far, and seem to be grateful for the time I spent explaining what we are planning and why. I think it also shows it's been worth doing it, if only to keep good karma and relationships.
I strongly believe in transparency and being candid about things like that. We might think that people will object but since we have good reasons for removal and we spent time on discussing and showing that, and we have stats backing up our data I think it's no brainer also for their community. I will wait until week passes and will start removal vote - unless something new comes up. J. On Mon, Nov 20, 2023 at 5:07 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > Looks like they would vote for removal too > <https://github.com/dask/community/issues/355#issuecomment-1816805948> > > On Fri, 17 Nov 2023 at 17:06, Jarek Potiuk <ja...@potiuk.com> wrote: > > > I was asked to open the issue in GitHub to get more visibility by Dask > > developers so here it is https://github.com/dask/community/issues/355 > > > > On Fri, Nov 17, 2023 at 1:21 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > OK. Seeing that - I think I will do the next step - I pointed this > > > discussion to at the discord of Dask and see if there is a volunteer > > there > > > who would like to take on fixing the "Quarantined" issue > > > https://github.com/apache/airflow/issues/32778 -> we have the flaky > > tests > > > currently marked as "Quarantined - so it is not really running in > regular > > > tests anyway. If there is no-one stepping in and wanting to commit to > > take > > > over and manage fixing of similar issues, I will start a voting thread > > for > > > removal, > > > > > > For me that is an ultimate (and very concrete) "checkpoint" if the > Dask > > > community is up to the task in our repo. And if they are not - they are > > > always free to fork it and maintain their own provider. I think it's a > > very > > > fair approach and quite reasonable expectation - taking into account > that > > > we have so little usage and so many problems. > > > > > > I announced our intentions in this thread: > > > > > > https://dask.discourse.group/t/planned-removal-of-the-daskexecutor-provider-in-airflow/2328 > > > > > > I explained the option Dask Community has and gave them a week for > > > reaction. If no-one steps up and starts seriously diagnosing and fixing > > the > > > flaky issue - I will start the vote on removal in about a week. > > > > > > J. > > > > > > > > > > > > On Fri, Nov 17, 2023 at 10:28 AM Wei Lee <weilee...@gmail.com> wrote: > > > > > >> +1 for removal if there is no active maintainer on this one > > >> > > >> Best, > > >> Wei > > >> > > >> > On Nov 17, 2023, at 6:00 PM, Andrey Anshin < > andrey.ans...@taragol.is> > > >> wrote: > > >> > > > >> > +1 for suspend and after a while remove > > >> > > > >> > There is a small chance that things would improve after we suspended > > >> this > > >> > provider. > > >> > > > >> > However we do not have a lot of statistics, AFAIK (correct me if I'm > > >> wrong) > > >> > it is only two cases: > > >> > * 1 provider suspended in the past and restored (Yandex), there is > no > > >> > process of removal at that moment. > > >> > * 1 provider suspended and removed (Qubole) > > >> > > > >> > > > >> > ---- > > >> > Best Wishes > > >> > *Andrey Anshin* > > >> > > > >> > > > >> > > > >> > On Fri, 17 Nov 2023 at 12:17, Amogh Desai <amoghdesai....@gmail.com > > > > >> wrote: > > >> > > > >> >> Theres very little incentive in maintaining this if theres no one > > >> actively > > >> >> maintaining it. > > >> >> > > >> >> I am totally for the removal +1 > > >> >> > > >> >> > > >> >> > > >> >> On Fri, 17 Nov 2023 at 1:21 AM, Collin McNulty > > >> >> <col...@astronomer.io.invalid> > > >> >> wrote: > > >> >> > > >> >>> +1 for removal > > >> >>> > > >> >>> On Thu, Nov 16, 2023 at 1:43 PM Hussein Awala <huss...@awala.fr> > > >> wrote: > > >> >>> > > >> >>>>> we would do it branching off at the TAG when the last release > > >> >> happened > > >> >>>> and develop/release a fix from there. > > >> >>>> > > >> >>>> Since it would be possible to release security patches, +1 to > > remove > > >> >> it. > > >> >>>> > > >> >>>> On Thu 16 Nov 2023 at 21:22, Oliveira, Niko > > >> >> <oniko...@amazon.com.invalid > > >> >>>> > > >> >>>> wrote: > > >> >>>> > > >> >>>>> If no one comes forward willing to support the executor long > term > > >> I'm > > >> >>> +1 > > >> >>>>> for removal. > > >> >>>>> > > >> >>>>> ________________________________ > > >> >>>>> From: Vincent Beck <vincb...@apache.org> > > >> >>>>> Sent: Thursday, November 16, 2023 10:59:40 AM > > >> >>>>> To: dev@airflow.apache.org > > >> >>>>> Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend > > >> >> (Remove?) > > >> >>>>> Daskexecutor provider > > >> >>>>> > > >> >>>>> CAUTION: This email originated from outside of the organization. > > Do > > >> >> not > > >> >>>>> click links or open attachments unless you can confirm the > sender > > >> and > > >> >>>> know > > >> >>>>> the content is safe. > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > > >> >>> externe. > > >> >>>>> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si > vous > > ne > > >> >>>> pouvez > > >> >>>>> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > > >> >> certain > > >> >>>> que > > >> >>>>> le contenu ne présente aucun risque. > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> +1 for removal > > >> >>>>> > > >> >>>>> On 2023/11/16 18:54:15 Jarek Potiuk wrote: > > >> >>>>>> More detailed comparison: > > >> >>>>>> > > >> >>>>>> apache-airflow 2.7.* ~ 255.000 downloads/day > > >> >>>>>> apache-aurflow-provider-dask-executor ~ 900/ day > > >> >>>>>> > > >> >>>>>> This means that *apache-airflow-providers-daskexecutor * is > > >> >>> downloaded > > >> >>>> in > > >> >>>>>> less of *0.3 %* cases, comparing to *apache-airflow* > > >> >>>>>> > > >> >>>>>> I'd say it's negligible usage. > > >> >>>>>> > > >> >>>>>> My personal vote would go for immediate removal. > > >> >>>>>> > > >> >>>>>> WDYT? > > >> >>>>>> > > >> >>>>>> J. > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> On Wed, Nov 15, 2023 at 10:39 PM Jarek Potiuk < > ja...@potiuk.com> > > >> >>>> wrote: > > >> >>>>>> > > >> >>>>>>> > > >> >>>>>>> On Wed, Nov 15, 2023 at 10:20 PM Elad Kalif < > elad...@gmail.com> > > >> >>>> wrote: > > >> >>>>>>> > > >> >>>>>>>> Now that the code is in its own provider we can check the > > >> >> download > > >> >>>>> stats > > >> >>>>>>>> of > > >> >>>>>>>> the library via Pypi stats. > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> Good point but this is only an indication. > > >> >>>>>>> > > >> >>>>>>> Currently this is only for Airflow 2.7+ and it is a bit > > difficult > > >> >>> to > > >> >>>>>>> compare those. The "fixed" numbers are misleading (we still do > > >> >> not > > >> >>>> know > > >> >>>>>>> where the 400.000 downloads/day jump to almost 1Million / day > > >> >> came > > >> >>>> from > > >> >>>>>>> mid-October. > > >> >>>>>>> The ratio is now 1000/day (daskexecutor): 800.000 day > (airflow) > > >> >> -> > > >> >>>>> ~0.1 > > >> >>>>>>> %. But those are only downloads - and not whether it's used - > > >> >>> likely > > >> >>>>> there > > >> >>>>>>> are a number of cases where CI pulls "all" extras. > > >> >>>>>>> > > >> >>>>>>> J. > > >> >>>>>>> > > >> >>>>>> > > >> >>>>> > > >> >>>>> > > >> --------------------------------------------------------------------- > > >> >>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > >> >>>>> For additional commands, e-mail: dev-h...@airflow.apache.org > > >> >>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > >> For additional commands, e-mail: dev-h...@airflow.apache.org > > >> > > >> > > >
