Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Jarek Potiuk
Ok. Happy to move it back then :). No problem with that. According to rules of AIP-21 it should actually be: "*from airflow.providers.kubernetes.operators.pod import KubernetesPodOperator*" (Case 2A. (drop _operator in module name) + Case 5B. (keep Operator in class name). We can have more than j

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Ash Berlin-Taylor
+1 for Python and Bash being in the stock install -- they are just _so_ commonly used that I think it makes sense to keep them in the base install. (and the virtualenv module is not an onerous dep, not caused us any problems. Yet). Kubeneretes is also a slighlty funny one since the deps for tha

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Jarek Potiuk
> > Another question is operators like SlackWebHookOperator depends on > SimpleHTTPOperator ! Will this cause dependencies issues or with proper > versioning this should be OK ? > Very good question Kaxil! This is one of the reasons we do not want to make yet full AIP-8 implementation. There will

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Jarek Potiuk
On Mon, Nov 11, 2019 at 4:22 PM Kamil Breguła wrote: > One more question. Are you sure you want to move Python and Bash from > core? These are the elements that are installed in every environment > because they are required by Airflow, so moving them to a separate > installed package is pointle

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Kaxil Naik
Another question is operators like SlackWebHookOperator depends on SimpleHTTPOperator ! Will this cause dependencies issues or with proper versioning this should be OK ? On Mon, Nov 11, 2019 at 3:22 PM Kamil Breguła wrote: > One more question. Are you sure you want to move Python and Bash from

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Kamil Breguła
One more question. Are you sure you want to move Python and Bash from core? These are the elements that are installed in every environment because they are required by Airflow, so moving them to a separate installed package is pointless in my opinion. On Mon, Nov 11, 2019 at 3:07 PM Kaxil Naik

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Kaxil Naik
I am fine with this list +1 On Mon, Nov 11, 2019 at 1:27 PM Jarek Potiuk wrote: > I am all for it Kamil! > > Super happy to treat Apache projects in the same way as "proprietary" > providers :). Anyone else has some other comments ? > > J. > > On Mon, Nov 11, 2019 at 2:17 PM Kamil Breguła > wro

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Jarek Potiuk
I am all for it Kamil! Super happy to treat Apache projects in the same way as "proprietary" providers :). Anyone else has some other comments ? J. On Mon, Nov 11, 2019 at 2:17 PM Kamil Breguła wrote: > I looked at this list and I'm only worried about two operators. > > airflow.contrib.operato

Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

2019-11-11 Thread Kamil Breguła
I looked at this list and I'm only worried about two operators. airflow.contrib.operators.vertica_to_hive airflow.contrib.operators.s3_to_hive If we want the operators to be grouped according to destination, then this operator should be in apache package. It is the members of the Apache community

Re: Donating code to add Common Workflow Language import to Airflow

2019-11-11 Thread Jarek Potiuk
Hello Andrey, I think both myself and Maxime - we asked some important questions. If you want to proceed with the donation, I think it would be great if you let us know what do you think about the issues we mentioned. I know also Michael whom I met at the workshops in Berlin - was very interested