Re: Airflow 1.8.0 alpha 4

2017-01-12 Thread Chris Riccomini
e all pending DAG runs in a single > >>> scheduler > >>>>> loop” from the 1.8.0 release (updated: https://github.com/apache/ > >>>>> incubator-airflow/pull/1980 <https://github.com/apache/ > >>>>> incubator-airflow/pull/1980&

Re: Airflow 1.8.0 alpha 4

2017-01-12 Thread Bolke de Bruin
-airflow/pull/1906 <https://github.com/apache/ >>>>> incubator-airflow/pull/1906>). The reason for this is that it, imho, >>>>> biases the scheduler towards a single DAG as it fills the queue with >>> tasks >>>>> from one DAG and the

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Dan Davydov
gt;> biases the scheduler towards a single DAG as it fills the queue with > > tasks > > >> from one DAG and then goes to the next DAG. Starving DAGs that come > > after > > >> the first for resources. As such it should be updated and that will &

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Chris Riccomini
then goes to the next DAG. Starving DAGs that come > after > >> the first for resources. As such it should be updated and that will take > >> time. > >> > >> Please let me know if I am incorrect. > >> > >> Thanks > >> Bo

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Bolke de Bruin
>> the first for resources. As such it should be updated and that will take >> time. >> >> Please let me know if I am incorrect. >> >> Thanks >> Bolke >> >>> On 10 Jan 2017, at 09:25, Bolke de Bruin <bdbr...@gmail.com&

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Chris Riccomini
eue with tasks from one DAG and then goes to the next DAG. > > Starving DAGs that come after the first for resources. As such it should > be > > updated and that will take time. > > > > Please let me know if I am incorrect. > > > > Thanks > > Bolke > &g

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Alex Van Boxel
e de Bruin <bdbr...@gmail.com> wrote: > > > > Dear All, > > > > I have made Airflow 1.8.0 alpha 4 available at > https://people.apache.org/~bolke/ <https://people.apache.org/~bolke/> . > Again no Apache release yet - this is for testing purposes. I consider t

Re: Airflow 1.8.0 alpha 4

2017-01-11 Thread Bolke de Bruin
As such it should be updated and that will take time. Please let me know if I am incorrect. Thanks Bolke > On 10 Jan 2017, at 09:25, Bolke de Bruin <bdbr...@gmail.com> wrote: > > Dear All, > > I have made Airflow 1.8.0 alpha 4 available at > https://people.apache.org/

Airflow 1.8.0 alpha 4

2017-01-10 Thread Bolke de Bruin
Dear All, I have made Airflow 1.8.0 alpha 4 available at https://people.apache.org/~bolke/ <https://people.apache.org/~bolke/> . Again no Apache release yet - this is for testing purposes. I consider this Alpha to be a Beta if not for the pending features. If the pending features are