Aye to that. B.
Sent from my iPhone > On 12 Apr 2019, at 17:17, Ash Berlin-Taylor <a...@apache.org> wrote: > > Somewhat off topic from the topic of just moving the well tested parts out of > contrib/ but okay. I thought I had somewhere but I can't find where I did it > before. > > Quickly: > > - How do we go about testing changes to core against hook? > - Who/what/when releases the split-out hooks. Are they ASF releases or > community. (If community then this can happen right now. Someone can just > create a package, publish on pypi and we can start pointing at that) > - What about CI? How do we test changes to core against operators? When? How > often? > - Some hooks need tighter integration (GCP,S3, Azure, ElasticSearch) as they > can be used by loggers. How do we do that? > - A "operator" package should probably depend upon Airflow to get > "BaseOperator" class etc, but that means that installing the > `apache-airflow[extra]` can't work as that would be a loop. Maybe we need an > `apache-airflow-core` instead. > > In short: I like "mono" repo as it has lots of advantages :) > > -ash > > >> On 12 Apr 2019, at 16:10, Chen Tong <cix...@gmail.com> wrote: >> >> Hi Ash, >> >> Could you state the questions you are concerned about? We may discuss it in >> this thread. >> >> Thank you! >> >>> On Fri, Apr 12, 2019 at 11:06 AM Ash Berlin-Taylor <a...@apache.org> wrote: >>> >>> Speaking as a release manager AIP-8 also had some large unanswered >>> workflow questions before I'd be happy about to "accept" it. >>> >>> -ash >>> >>>> On 12 Apr 2019, at 16:04, Chen Tong <cix...@gmail.com> wrote: >>>> >>>> AIP-8 is intended to achieve this by separate operators/hooks to >>> individual >>>> packages. But the author hasn't been actively working on this recently. >>>> >>>> On Fri, Apr 12, 2019 at 9:29 AM Julian De Ruiter < >>>> julianderui...@godatadriven.com> wrote: >>>> >>>>> Isn’t this in contradiction with AIP-8, which is aimed at removing >>>>> operators/hooks from the core Airflow package? >>>>> >>>>> Personally I would rather remove hooks/operators from Airflow than add >>>>> even more to the Airflow core. This counts double for the contrib stuff, >>>>> which is often poorly designed and/or tested. >>>>> >>>>> Best, >>>>> Julian >>>>> >>>>>> On 12 Apr 2019, at 10:23, Bolke de Bruin <bdbr...@gmail.com> wrote: >>>>>> >>>>>> That’s perfectly fine to me. >>>>>> >>>>>> Verstuurd vanaf mijn iPad >>>>>> >>>>>>> Op 12 apr. 2019 om 10:20 heeft Kaxil Naik <kaxiln...@gmail.com> het >>>>> volgende geschreven: >>>>>>> >>>>>>> Ok. How about moving the properly tested and maintained hooks/ops from >>>>>>> contrib to core? >>>>>>> >>>>>>>> On Fri, Apr 12, 2019, 09:13 Bolke de Bruin <bdbr...@gmail.com> >>> wrote: >>>>>>>> >>>>>>>> I disagree. Core signals “properly tested” and maintained. Ie. A kind >>>>> of >>>>>>>> quality. I don’t think contrib has that. >>>>>>>> >>>>>>>> Verstuurd vanaf mijn iPad >>>>>>>> >>>>>>>>> Op 12 apr. 2019 om 10:03 heeft Kaxil Naik <kaxiln...@gmail.com> het >>>>>>>> volgende geschreven: >>>>>>>>> >>>>>>>>> Contrib folder was used when it was used at Airbnb. Currently, it >>>>> doesn't >>>>>>>>> make any sense and we have equal responsibility to maintain all the >>>>>>>> hooks, >>>>>>>>> operators, sensors in contrib folder as we do for core. >>>>>>>>> >>>>>>>>> I would suggest to remove contrib folder and move all hooks, ops, >>> and >>>>>>>>> sensors to the core folder. >>>>>>>>> >>>>>>>>> Or reorganize the folder structure similar to what was discussed in >>> a >>>>>>>>> mailing thread few months ago. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Kaxil >>>>>>>> >>>>> >>>>> >>> >>> >
