Re: Kubernetes executor production readiness?

2019-06-28 Thread Qingping Hou
On Wed, Jun 26, 2019 at 2:35 AM Ry Walker wrote: > The nice thing about k8s executor is that you can redeploy airflow without > the need to drain the system to ensure work won’t be affected. > > Also, going all k8s gives you autoscaling. > > If you’d like to try our both, you can install trial of

Re: [VOTE] Release Apache Airflow 1.10.4rc1 as 1.10.4

2019-06-28 Thread Deng Xiaodong
+1 (non-binding). There is a minor issue: after clicking the new “Toggle Wrap” button in DAG Code View page, the line numbers on the left side may become unaligned with the code lines. However it’s not a blocker at all to me. Thanks Ash for running the release again, and thanks to all the contr

Re: Travis builds in a queue for hours

2019-06-28 Thread Jarek Potiuk
Thanks Aizhamal! I spoke already to Gris and she confirmed that as well and the 8th of July date is ok for us as we will have to evaluate and prepare as well. Have a nice trip. J. On Fri, Jun 28, 2019 at 4:25 PM Aizhamal Nurmamat kyzy wrote: > Hi all, > > On Thu, Jun 27, 2019 at 15:28 Jarek Pot

Re: Travis builds in a queue for hours

2019-06-28 Thread Aizhamal Nurmamat kyzy
Hi all, On Thu, Jun 27, 2019 at 15:28 Jarek Potiuk wrote: > Yeah. I also have a working version of Cloud build configuration and we can > run the tests on cloud build if we can get some credits from Google. I can look into getting a small amount of credits approved for this, to see if it’s use

Re: Travis builds in a queue for hours

2019-06-28 Thread Jarek Potiuk
I had numerous issues related to particular backends. If you go deeper, i even had Postgres-related issue which only occurred on python3.6 (python 3.5 worked fine. It was related to parsing DB URL :) ). For sure testing all matrix combination is an overkill, but we need to test all backends. I also

Re: Travis builds in a queue for hours

2019-06-28 Thread Philippe Gagnon
I concur. There are subtle differences in SQLAlchemy behavior depending on the backend... In that context testing against multiple DBMS makes sense and adds value. @Conrad: I do see your point though, testing libraries should not be Airflow's responsibility but from a pragmatic point of view we sh

Re: Travis builds in a queue for hours

2019-06-28 Thread Driesprong, Fokko
Yes Conrad, this is actually true. However I think it is important to test different backends, as an example, we had a lot of issues with MySQL's timezone behavior, besides that, it would be possible to test specific database evolution steps for certain backends. Cheers, Fokko Op vr 28 jun. 2019

Re: Travis builds in a queue for hours

2019-06-28 Thread Conrad Dean
Regarding the matrix of backends to test -- is this testing specific platform integrations, or if everything is routed through SQLAlchemy's ORM does this test suite just end up testing SQLAlchemy? It makes sense to have the ability to swap backends out for the sake of reproducing issues when somet

[VOTE] Release Apache Airflow 1.10.4rc1 as 1.10.4

2019-06-28 Thread Ash Berlin-Taylor
Hey all, I have cut Airflow 1.10.4 RC1. This email is calling a vote on the release, which will last for 72 hours (2019-07-01 12:08 Z), and until three binding votes have been cast. Consider this my (binding) +1. Airflow 1.10.4 RC1 is available at: https://dist.apache.org/repos/dist/dev/airflo