KevinYang21 edited a comment on issue #5615: [AIRFLOW-5035] Remove multiprocessing.Manager in-favour of Pipes URL: https://github.com/apache/airflow/pull/5615#issuecomment-514809181 @ashb It is not a native metrics Airflow emits unfortunately, as it is an end to end metric that I need an actual task to help measure. It is from a metric DAG we setup internally, which send `actual execution time - expected execution time` to datadog. I recall seeing a PR emitting the delay within the scope of scheduler( I may be wrong tho, been while), it would be the ideal measurement we want if it indeed exists. Also I really appreciate that you would spend time dive deep into this issue. We saw this and just patch it with a restart script killing the orphan processes. I will definitely squeeze as much time as possible to help push this through( day time work has been a bit crazy these days :P).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
