As an Airflow user I would just like to add I strongly agree with this. The
current behavior is intuitive but documented incorrectly.
IMO cron notation is expected to work like cron on modern Linux/Unix, i.e.
whatever the system or user timezone is it follows the DST rules. And for jobs
you wis
Hey Sachin, Celery should take care of this kind of "warm shutdown" for you
-
https://docs.celeryproject.org/en/latest/userguide/workers.html#process-signals
.
If you bump the image tag on the worker Kubernetes Deployment to upgrade
something, kube/docker will send a SIGTERM to the worker pods, te
Hi,
I have a airflow setup with Celery executors on kubernetes. The cluster has
many workers pods picking jobs from different queues (rabbitmq).
I want some of my airflow worker pods to continue with jobs which it is
running currently and stop fetching/starting any new jobs from the queue.
On com
Would be great to have epics. Fokko has previously created a few issues with
sub-issues, that also helps organisation.
Bas
> On 13 May 2019, at 20:49, Daniel Imberman wrote:
>
> Hi everyone,
>
> I've been working on cleaning up the backlog of tickets for airflow, and
> I'm running into an iss
