GitHub user o-nikolas added a comment to the discussion: Airflow 3.1.5 Mixed Executors - Task Instance Not Found & State Desync Issues
I see you're using the config `AIRFLOW__CELERY_KUBERNETES_EXECUTOR__KUBERNETES_QUEUE=kubernetes` This is for the old (deprecated) Celery/k8s hybrid executor, not to be used with multiple executor configuration. I'm wondering if you have a setup that is half converted to the modern multiple executor config. > Is the per-task executor= parameter fully supported in Airflow 3.x mixed > executor setups? The documentation suggests it is (since 2.6+), but we're experiencing critical failures. Multiple executor config was released in 2.10, where in the docs do you see it say this should work with 2.6? I'm wondering if you're mixing configuration between versions. Lastly, I know that Celery can be quite unstable. With tasks being lost in some circumstances, or issues with the task SDK API (task tokens expiring by the time they reach the worker). It's worth looking more into this, which is unrelated to multiple executor configuration. GitHub link: https://github.com/apache/airflow/discussions/61468#discussioncomment-15729791 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
