GitHub user hss-rmatthews created a discussion: Airflow 3.0.6 - apache-airflow-providers-celery 3.12.2 - Process 'ForkPoolWorker-' exited with 'signal 9 (SIGKILL)
Hi All, We are running Airflow version 3.0.6 in MWAA and have a task failing without any logs showing in Airflow. The task log shows: An error occurred (ResourceNotFoundException) when calling the GetLogEvents operation: The specified log stream does not exist. However, when investigating the worker logs we are finding the following entries: [error] Timed out waiting for UP message from <ForkProcess(ForkPoolWorker-245, started daemon)> [celery.concurrency.asynpool] [ERROR/MainProcess] Process 'ForkPoolWorker-245' pid: exited with 'signal 9 (SIGKILL)' This particular task is using the S3ToRedshiftOperator and I have verified that the command is actually present and in our Redshift logs and it has succeeded. The spec of the environment in MWAA is: mw1.medium with a maximum worker count of 20. We currently only have a single DAG running once a day (as we are in the process of migrating from version 2.10.3). Thanks Rob GitHub link: https://github.com/apache/airflow/discussions/62465 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
