Thanks Stefan for picking this up. The sensors are in desperate need for some redesign for the aforementioned reasons. Please note this ticket: https://issues.apache.org/jira/browse/AIRFLOW-2001 It addresses the same issue.
Regarding the open question. I would be reluctant for introducing new states. Adding new states involves also changing/adding logic in the scheduler. This scheduler is already far too complex right now. Maybe we can also do something with the priority of the sensor, maybe lower the priority once it has been polled, and came back with a negative state. In such a strategy the other tasks will get priority over the sensors. Cheers, Fokko 2018-07-12 14:53 GMT+02:00 Stefan Seelmann <m...@stefan-seelmann.de>: > Hi all, > > I'd like to discuss a proposal to enable explicit re-scheduling of > sensors. I think there is demand for such a thing, in the last weeks > multiple people asked for it or mentioned workarounds. > > I created a Jira [1] that describes the proposal and an initial PR [2]. > > Feedback welcomed :-) > > Kind Regards, > Stefan > > [1] https://issues.apache.org/jira/browse/AIRFLOW-2747 > [2] https://github.com/apache/incubator-airflow/pull/3596 >
