eladkal commented on pull request #17576:
URL: https://github.com/apache/airflow/pull/17576#issuecomment-916429784
> I think that should be explicit in the dag
+1
> I hadn't considered something like "give my stream processing up to X
minutes to process the data for this interval and make it available".
But I think that should be explicit in the dag. If this task should wait
until 30mins after the interval is over, then to make it easy to reason about
that should be a (Async)TimeDeltaSensor.
Yeah that works. but also isn't it a natural extension for trigger rules?
for now trigger rules verify only against upstream tasks status. Would it be
reasonable to extend it also to support rules of other areas like timedelta
from upstream tasks?
Something like: ` DummyOperator(..., trigger_rule={"state": "all_done",
"delay": 30})`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
