Thanks a lot.
On Tue, Sep 11, 2018, 6:25 PM ramandu...@gmail.com
wrote:
> We use max_threads = number of scheduler cores.
>
> On 2018/09/11 09:49:53, Chandu Kavar wrote:
> > Thanks Raman,
> >
> > Understood.
> >
> > We have around 500 DAGs. What value do you suggest for max_threads?
> >
> > On
We use max_threads = number of scheduler cores.
On 2018/09/11 09:49:53, Chandu Kavar wrote:
> Thanks Raman,
>
> Understood.
>
> We have around 500 DAGs. What value do you suggest for max_threads?
>
> On Tue, Sep 11, 2018, 5:44 PM ramandu...@gmail.com
> wrote:
>
> > Hi Chandu,
> > How many
Thanks Raman,
Understood.
We have around 500 DAGs. What value do you suggest for max_threads?
On Tue, Sep 11, 2018, 5:44 PM ramandu...@gmail.com
wrote:
> Hi Chandu,
> How many dag files are there on the scheduler. As per my understanding
> scheduler processes each file to trigger any dag/task
Hi Chandu,
How many dag files are there on the scheduler. As per my understanding
scheduler processes each file to trigger any dag/task run. It spawns number of
processes equivalent to "max_threads" count to parallelize file processing. So
you can try by increasing airflow config's max_threads
Hi All,
We are seeing there is a delay between triggering the next task. Please
find the attached screenshot.
First and the last task are very small. So, you can see the tiny object.
[image: image.png]
Do you suggest any airflow config can resolve this problem?
Thanks,
Chandu
