Hi, I have a airflow setup with Celery executors on kubernetes. The cluster has many workers pods picking jobs from different queues (rabbitmq).
I want some of my airflow worker pods to continue with jobs which it is running currently and stop fetching/starting any new jobs from the queue. On completion of the running job, I will restart the worker pod. Rest of the airflow worker pods in the cluster will continue with running jobs and starting new jobs. Is there a way to achive this? Thanks, Sachin