[
https://issues.apache.org/jira/browse/AIRFLOW-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beau Barker updated AIRFLOW-3102:
---------------------------------
Description:
I am finding tasks aren't starting for me.
The scheduler logs are showing "Processor exited with return code -9".
{code:java}
airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process
(PID: 23) to generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
airflow.jobs.SchedulerJob INFO:
================================================================================
DAG File Processing Stats
File Path PID Runtime Last Runtime Last Run
----------------------------------------------------------- ----- ---------
-------------- -------------------
/usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s 2018-09-22T01:27:53
================================================================================
airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for
/usr/local/airflow/dags/definitions/my_dag.py finished
airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for
/usr/local/airflow/dags/definitions/my_dag.py exited with return code -9.
{code}
After much searching I've narrowed it down to a line in logging_mixin.py:
{code:java}
self.logger.log(self.level, self._buffer)
{code}
Link on Github:
https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/logging_mixin.py#L95
If I comment that line, the DagFileProcessorManager succeeds and tasks do start.
was:
I am finding tasks aren't starting for me.
The scheduler logs are showing "Processor exited with return code -9".
{code:java}
airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process
(PID: 23) to generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
airflow.jobs.SchedulerJob INFO:
================================================================================
DAG File Processing Stats
File Path PID Runtime Last Runtime Last Run
----------------------------------------------------------- ----- ---------
-------------- -------------------
/usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s 2018-09-22T01:27:53
================================================================================
airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for
/usr/local/airflow/dags/definitions/my_dag.py finished
airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for
/usr/local/airflow/dags/definitions/my_dag.py exited with return code -9.
{code}
After much searching I've narrowed it down to [this line in
jobs.py|https://github.com/apache/incubator-airflow/blob/v1-10-stable/airflow/jobs.py#L372]:
{code:java}
sys.stdout = stdout
{code}
If I comment that lines, the DagFileProcessorManager succeeds and tasks do
start.
(Still investigating)
> Processor exited with return code -9
> ------------------------------------
>
> Key: AIRFLOW-3102
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3102
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.10.0
> Environment: Docker
> Reporter: Beau Barker
> Priority: Major
>
> I am finding tasks aren't starting for me.
> The scheduler logs are showing "Processor exited with return code -9".
>
> {code:java}
> airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process
> (PID: 23) to generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
> airflow.jobs.SchedulerJob INFO:
> ================================================================================
> DAG File Processing Stats
> File Path PID Runtime Last Runtime Last Run
> ----------------------------------------------------------- ----- ---------
> -------------- -------------------
> /usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s
> 2018-09-22T01:27:53
> ================================================================================
> airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for
> /usr/local/airflow/dags/definitions/my_dag.py finished
> airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for
> /usr/local/airflow/dags/definitions/my_dag.py exited with return code -9.
> {code}
> After much searching I've narrowed it down to a line in logging_mixin.py:
> {code:java}
> self.logger.log(self.level, self._buffer)
> {code}
> Link on Github:
> https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/logging_mixin.py#L95
> If I comment that line, the DagFileProcessorManager succeeds and tasks do
> start.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
