Re: [VOTE] AIP-21 update for Airflow 1.10.* backportability

Re For example S3 -> GCS should be in "google" provider, but GCS-> S3 should > be in "amazon > So if there were a BigQueryToS3 or SnowflakeToS3 operator, would you put this in AWS? I feel like storage should be a secondary consideration concerning object naming. Using snowflake as an example, w

[VOTE] AIP-21 update for Airflow 1.10.* backportability

Hello Airflow Community, The email calls for a vote to update AIP-21 Changes in import paths with the changes described below. The vote will last till Saturday 8th 2am CEST (72 hours). Committers have a binding

Re: [PROPOSE] Ease future migration path to 2.0 by provider's operators/hook backporting to 1.10.*

Thanks Ash! It seems it works really well and is super simple! I have a POC working for Airflow: https://github.com/apache/airflow/pull/6507 I managed to build and pip-install two packages: 1) apache_airflow 2.0 -> which is the same as today - containing everything - including providers and gcp.

Re: Wrong Urllib/Urlquote imports in v1-10-test

Cool. Great that it works OK :) As I mentioned in the comment in github: I think it's still worth changing the way I did and back-porting isort changes. Having the local py2/py3 urlparse imports in v1-10-test make them more isort-friendly and better for future cherry-picks. Maybe not nice but we h

Re: Wrong Urllib/Urlquote imports in v1-10-test

In [9]: urllib.parse Out[9]: So it's coming form the future module we use in py2. So nothing is broken thankfully. > On 5 Nov 2019, at 14:58, Jarek Potiuk wrote: > > While cherry-picking some of the latest "prod-image" related changes to > v1-10-test I also cherry-picked the 'isort' change

Wrong Urllib/Urlquote imports in v1-10-test

While cherry-picking some of the latest "prod-image" related changes to v1-10-test I also cherry-picked the 'isort' change - this way it will be a bit easier to cherry-pick any changes that were done after the isort. While doing it I found that we had some python3-only imports in some of the files

[upcoming event] Data Orchestration Summit

Hey, Quick last minute note about a somewhat-Airlfow-related event coming up: the Data Orchestration Summit (@ Computer History Museum in Mountain View) this Thursday. I'm scheduled to be on a panel about creating open source projects and co

Re: [PROPOSE] Ease future migration path to 2.0 by provider's operators/hook backporting to 1.10.*

Yes let's just do (1) for now. On Tue, Nov 5, 2019, 08:48 Jarek Potiuk wrote: > Thanks Ash! It might indeed work. I will take it from there and try to make > a POC PR with airflow. > > It's a bit different approach than google-python libraries (they keep all > the libraries as separate sub-pac

Re: [PROPOSE] Ease future migration path to 2.0 by provider's operators/hook backporting to 1.10.*

Thanks Ash! It might indeed work. I will take it from there and try to make a POC PR with airflow. It's a bit different approach than google-python libraries (they keep all the libraries as separate sub-packages/mini projects inside the main project). The approach you propose is far less invasive