GitHub user budaesandrei added a comment to the discussion: Airflow 3.0.6 - apache-airflow-providers-celery 3.12.2 - Process 'ForkPoolWorker-' exited with 'signal 9 (SIGKILL)
I'm having this too and struggling to find the proper fix. I've tried reducing my **celery.worker_autoscale** from "20,0" -> "10,0"...I tried adding **core.execute_tasks_new_python_interpreter** = true. Supposedly that tells celery to start new python interpreters instead of using **os.fork()** which seems to be the actual issue in **python3.12** but that didn't work either. I'm running out of ideas what to try. Even my agent is just saying "you have no option but to wait for AWS to upgrade to 3.1+ with the fix" which is not an option for us using MWAA. And we can't downgrade because we'd lose data. GitHub link: https://github.com/apache/airflow/discussions/62465#discussioncomment-15974300 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
