uranusjr commented on issue #38632: URL: https://github.com/apache/airflow/issues/38632#issuecomment-2031443184
> Just to confirm, does a DEFFERED task really transition directly to RUNNING? I didn’t say that; I only said a task transitions _from_ RUNNING to DEFERRED 😛 A deferred task is indeed set to SCHEDULED by the triggerer, when the trigger is fulfilled, to be handed back to the scheduler; in some cases the handed back ti isn’t even transitioned back to RUNNING at all, but to a finished state directly (when things time out etc.) > the current desired behavior of start_date is [snip] Yeah I think that’s about right. UP_FOR_RESCHEDULE is technically a restart because a ti set to it will completely go through the worker cycle again. But again it is special (in a bad way) and let’s try to just not change it. > According to my referenced code, ti.set_state(TaskInstanceState.UP_FOR_RETRY) actually sets current time as ti.end_date (because of or ti.state == TaskInstanceState.UP_FOR_RETRY). However, from reading your commentary, this may also be the wrong behavior. Ah no, my comment on this was wrong, actually. I was confusing UP_FOR_RETRY and UP_FOR_RESCHEDULE. And although we can technically argue UP_FOR_RETRY maybe should not set end_date, [it needs to do so because retry in Airflow depends on a task instance to have an end_date](https://github.com/apache/airflow/commit/8291fabaf95dd4dc2ff2ab33be07ccd1a34df2ae). Also note that [there’s currently a discussion on keeping task instance history across tries](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-64%3A+Keep+TaskInstance+try+history), so instead of overwriting those dates on retry, we will have a separate row in the database, each having its start and end dates, thus changing the situation a bit. --- By the way, I want to specifically point out the work you put in this. It is a mess where Airflow sets ti states in various places, and it’s evident you must have traced a lot of code to reach the understanding on the issue. Thanks for putting in the effort. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
