Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-20 Thread Pete DeJoy
Just a quick addition to Ry’s instructions- you’ll need to run `astro dev init` between steps 1 and 2 to initiate an Airflow project and generate the necessary skeleton files to run locally (including the Dockerfile). Hope folks find this process helpful- really exciting to see the alpha build r

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-20 Thread Jarek Potiuk
Cool! On Tue, Oct 20, 2020 at 3:55 PM Ry Walker wrote: > FYI we've also made it easy to run Airflow 2 alpha locally using the astro > CLI. > > 1. Install the CLI: > > curl -sSL https://install.astronomer.io | sudo bash -s -- v0.21.0 > > 2. Change Dockerfile to: > > FROM astronomerio/ap-airflow:2

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-20 Thread Ry Walker
FYI we've also made it easy to run Airflow 2 alpha locally using the astro CLI. 1. Install the CLI: curl -sSL https://install.astronomer.io | sudo bash -s -- v0.21.0 2. Change Dockerfile to: FROM astronomerio/ap-airflow:2.0.0-1.dev3-buster-onbuild 3. You can install packages in requirements.tx

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-20 Thread Jarek Potiuk
Sure. You should install the .whl packages directly via `pip install <>.whl`. For now we do not have yet PIP-released version so you have to manually choose the right extras when you install airflow and then install the provider: `pip install 'apache_airflow-2.0.0a1-py3-none-any.whl[google]'

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-20 Thread Julian De Ruiter
Hi Jarek, Can you maybe provide some guidelines on how to install these provider packages in the current alpha? Tried some things on my own, but seem to be running into issues. Best, Julian On 2020/10/14 07:00:08, Jarek Potiuk wrote: > A small follow up: The 2.0.0a1 release is the "core" rel

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-19 Thread Jarek Potiuk
Few points to add from my side to discuss today - to plan the work for the upcoming week: * For 1st Beta I think it will be important to release PyPI packages I believe * I think the automated providers registering might be partially ready as well for the beta, probably not all of the features - I

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-18 Thread Jarek Potiuk
The usual pre-meeting summary from my side. In preparation for Airflow 2.0 Dev call tomorrow, I have prepared some bug fixes and improvements to the providers approach. We are steadily moving in the mini-project https://github.com/apache/airflow/projects/5 . Some of them already merged (thanks tho

Re: Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-14 Thread Jarek Potiuk
A small follow up: The 2.0.0a1 release is the "core" release only. It has no "providers" installed. Airflow 2.0 will be distributed as a number of separate packages: "core" will be released separately and each of the providers has its own package to install. Once we release it in PyPI, the right pr

Airflow 2.0.0.alpha1 snapshot ready for testing!

2020-10-13 Thread Ash Berlin-Taylor
I'm proud to announce the availability of Apache Airlow 2.0.0.alpha1 for testing! First the caveat: this is an alpha release. Do not run it in production, it might not be without serious problems, and in the extreme case you may have to reset your database between this and the beta or release c