[ https://issues.apache.org/jira/browse/AIRFLOW-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ash Berlin-Taylor resolved AIRFLOW-2895. ---------------------------------------- Resolution: Fixed Fix Version/s: 1.10.1 > Prevent scheduler from spamming heartbeats/logs > ----------------------------------------------- > > Key: AIRFLOW-2895 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2895 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler > Reporter: Dan Davydov > Assignee: Dan Davydov > Priority: Major > Fix For: 1.10.1 > > > There seems to be a couple of problems with > [https://github.com/apache/incubator-airflow/pull/2986] that cause the sleep > to not trigger and Scheduler heartbeating/logs to be spammed: > # If all of the files are being processed in the queue, there is no sleep > (can be fixed by sleeping for min_sleep even if there are no files) > # I have heard reports that some files can return a parsing time that is > monotonically increasing for some reason (e.g. file actually parses in 1s > each loop, but the reported duration seems to use the very time the file was > parsed as the start time instead of the last time), I haven't confirmed this > but it sounds problematic. > To unblock the release I'm reverting this PR for now. It should be re-added > with tests/mocking. -- This message was sent by Atlassian JIRA (v7.6.3#76005)