dabla commented on issue #44618: URL: https://github.com/apache/airflow/issues/44618#issuecomment-2572818531
> Btw, I just (locally) implemented your refactor and the separation between the 2 flows (refresh trigger and get refresh status) in 2 deferrable triggers seems to let enough time between the refresh creation and the first "get refresh status" request. It seems that I don't get the error anymore (not that the bug is really fixed, but in my environment, the duration between both events is enough to prevent the error). > > > the operator can then retry the second case directly instead of redoing the whole flow > > Regarding this, how/where would you retry the second case? Indeed, that's also a consequence of using the trigger twice, as more time will pass automatically between both invocations, the error will probably not occurs anymore and everything will succeed in one attempt. For the second case, in the operator, you should do in the execute method an xcom_pull to see if there is an existing dataset_refresh_id or not, and if not you know you have to execute the whole flow, if it's there then you know you should only trigger the second part. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
