Hi all, We have made the decision to migrate all our existing ETL jobs to Airflow. Right now, we are deciding whether celery or kubernetes executor would be a better fit.
Is anyone currently using kubernetes executor in production? If so, how well does it scale and how stable is it compared to celery executor? We are planning to run about 2000 tasks per day to start with. >From what I have seen so far, with kubernetes executor, we will be able to restrict resource usage for each task run. Can anyone share some insights on what other advantages does celery executor provide over kubernetes executor except maturity? Thanks, QP
