Looks like they would vote for removal too <https://github.com/dask/community/issues/355#issuecomment-1816805948>
On Fri, 17 Nov 2023 at 17:06, Jarek Potiuk <[email protected]> wrote: > I was asked to open the issue in GitHub to get more visibility by Dask > developers so here it is https://github.com/dask/community/issues/355 > > On Fri, Nov 17, 2023 at 1:21 PM Jarek Potiuk <[email protected]> wrote: > > > OK. Seeing that - I think I will do the next step - I pointed this > > discussion to at the discord of Dask and see if there is a volunteer > there > > who would like to take on fixing the "Quarantined" issue > > https://github.com/apache/airflow/issues/32778 -> we have the flaky > tests > > currently marked as "Quarantined - so it is not really running in regular > > tests anyway. If there is no-one stepping in and wanting to commit to > take > > over and manage fixing of similar issues, I will start a voting thread > for > > removal, > > > > For me that is an ultimate (and very concrete) "checkpoint" if the Dask > > community is up to the task in our repo. And if they are not - they are > > always free to fork it and maintain their own provider. I think it's a > very > > fair approach and quite reasonable expectation - taking into account that > > we have so little usage and so many problems. > > > > I announced our intentions in this thread: > > > https://dask.discourse.group/t/planned-removal-of-the-daskexecutor-provider-in-airflow/2328 > > > > I explained the option Dask Community has and gave them a week for > > reaction. If no-one steps up and starts seriously diagnosing and fixing > the > > flaky issue - I will start the vote on removal in about a week. > > > > J. > > > > > > > > On Fri, Nov 17, 2023 at 10:28 AM Wei Lee <[email protected]> wrote: > > > >> +1 for removal if there is no active maintainer on this one > >> > >> Best, > >> Wei > >> > >> > On Nov 17, 2023, at 6:00 PM, Andrey Anshin <[email protected]> > >> wrote: > >> > > >> > +1 for suspend and after a while remove > >> > > >> > There is a small chance that things would improve after we suspended > >> this > >> > provider. > >> > > >> > However we do not have a lot of statistics, AFAIK (correct me if I'm > >> wrong) > >> > it is only two cases: > >> > * 1 provider suspended in the past and restored (Yandex), there is no > >> > process of removal at that moment. > >> > * 1 provider suspended and removed (Qubole) > >> > > >> > > >> > ---- > >> > Best Wishes > >> > *Andrey Anshin* > >> > > >> > > >> > > >> > On Fri, 17 Nov 2023 at 12:17, Amogh Desai <[email protected]> > >> wrote: > >> > > >> >> Theres very little incentive in maintaining this if theres no one > >> actively > >> >> maintaining it. > >> >> > >> >> I am totally for the removal +1 > >> >> > >> >> > >> >> > >> >> On Fri, 17 Nov 2023 at 1:21 AM, Collin McNulty > >> >> <[email protected]> > >> >> wrote: > >> >> > >> >>> +1 for removal > >> >>> > >> >>> On Thu, Nov 16, 2023 at 1:43 PM Hussein Awala <[email protected]> > >> wrote: > >> >>> > >> >>>>> we would do it branching off at the TAG when the last release > >> >> happened > >> >>>> and develop/release a fix from there. > >> >>>> > >> >>>> Since it would be possible to release security patches, +1 to > remove > >> >> it. > >> >>>> > >> >>>> On Thu 16 Nov 2023 at 21:22, Oliveira, Niko > >> >> <[email protected] > >> >>>> > >> >>>> wrote: > >> >>>> > >> >>>>> If no one comes forward willing to support the executor long term > >> I'm > >> >>> +1 > >> >>>>> for removal. > >> >>>>> > >> >>>>> ________________________________ > >> >>>>> From: Vincent Beck <[email protected]> > >> >>>>> Sent: Thursday, November 16, 2023 10:59:40 AM > >> >>>>> To: [email protected] > >> >>>>> Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [DISCUSS] Suspend > >> >> (Remove?) > >> >>>>> Daskexecutor provider > >> >>>>> > >> >>>>> CAUTION: This email originated from outside of the organization. > Do > >> >> not > >> >>>>> click links or open attachments unless you can confirm the sender > >> and > >> >>>> know > >> >>>>> the content is safe. > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > >> >>> externe. > >> >>>>> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous > ne > >> >>>> pouvez > >> >>>>> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > >> >> certain > >> >>>> que > >> >>>>> le contenu ne présente aucun risque. > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> +1 for removal > >> >>>>> > >> >>>>> On 2023/11/16 18:54:15 Jarek Potiuk wrote: > >> >>>>>> More detailed comparison: > >> >>>>>> > >> >>>>>> apache-airflow 2.7.* ~ 255.000 downloads/day > >> >>>>>> apache-aurflow-provider-dask-executor ~ 900/ day > >> >>>>>> > >> >>>>>> This means that *apache-airflow-providers-daskexecutor * is > >> >>> downloaded > >> >>>> in > >> >>>>>> less of *0.3 %* cases, comparing to *apache-airflow* > >> >>>>>> > >> >>>>>> I'd say it's negligible usage. > >> >>>>>> > >> >>>>>> My personal vote would go for immediate removal. > >> >>>>>> > >> >>>>>> WDYT? > >> >>>>>> > >> >>>>>> J. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> On Wed, Nov 15, 2023 at 10:39 PM Jarek Potiuk <[email protected]> > >> >>>> wrote: > >> >>>>>> > >> >>>>>>> > >> >>>>>>> On Wed, Nov 15, 2023 at 10:20 PM Elad Kalif <[email protected]> > >> >>>> wrote: > >> >>>>>>> > >> >>>>>>>> Now that the code is in its own provider we can check the > >> >> download > >> >>>>> stats > >> >>>>>>>> of > >> >>>>>>>> the library via Pypi stats. > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> Good point but this is only an indication. > >> >>>>>>> > >> >>>>>>> Currently this is only for Airflow 2.7+ and it is a bit > difficult > >> >>> to > >> >>>>>>> compare those. The "fixed" numbers are misleading (we still do > >> >> not > >> >>>> know > >> >>>>>>> where the 400.000 downloads/day jump to almost 1Million / day > >> >> came > >> >>>> from > >> >>>>>>> mid-October. > >> >>>>>>> The ratio is now 1000/day (daskexecutor): 800.000 day (airflow) > >> >> -> > >> >>>>> ~0.1 > >> >>>>>>> %. But those are only downloads - and not whether it's used - > >> >>> likely > >> >>>>> there > >> >>>>>>> are a number of cases where CI pulls "all" extras. > >> >>>>>>> > >> >>>>>>> J. > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> >>>>> > >> --------------------------------------------------------------------- > >> >>>>> To unsubscribe, e-mail: [email protected] > >> >>>>> For additional commands, e-mail: [email protected] > >> >>>>> > >> >>>>> > >> >>>> > >> >>> > >> >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> >
