argibbs commented on issue #34339: URL: https://github.com/apache/airflow/issues/34339#issuecomment-1718280778
> For your last guess, where do you think the timeout is? > > * is it in the dag processing job, and because of this timeout some other jobs (mainly the executor job) are killed? > * or it's in the executor's job that has to load the dagbag from the dag file to get the dag, and it fails when the processing time exceeds the timeout? The error is very explicitly coming from here: https://github.com/apache/airflow/blob/main/airflow/dag_processing/manager.py#L1215-L1219 and is inside the dag processing job. i.e. in dag_processor_manager.log. We have actually already set `AIRFLOW__CORE__DAGBAG_IMPORT_TIMEOUT` quite high (not like hours, but multiple minutes)... so the latter is very definitely not failing. The fix I'm attempting is similarly crude, but just to increase `AIRFLOW__CORE__DAG_FILE_PROCESSOR_TIMEOUT` to several minutes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
