Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
nathadfield commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2568848416 @dstandish Are you able to comment on this given you worked on #33718? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
kien-truong commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2568637999 It's partially overlap with [#33718](https://github.com/apache/airflow/pull/33718), which is making Deferrable sensor to not retry when `timeout` is reach. However, even with that fix in place, the sensor still needs to call defer with the correct `timeout` parameter, which is not the case with `ExternalTaskSensor`: it's passing `execution_timeout` instead of `timeout` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
nathadfield commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2567874193 Actually, unless I'm missing some other issue, I believe this is a duplication of a previously logged issue and looks to be addressed in the forthcoming [Airflow 2.11](https://github.com/apache/airflow/pull/33718). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
seifrajhi commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2567860555 Stills relevant, to remove stale label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
github-actions[bot] commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2566027075 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
nathadfield commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2545507371 I think it is correct that sensors that have exhausted their timeout should not retry; instead the timeout should be longer to cover a wide enough window of waiting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
nathadfield commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2545500341 There is currently a behavioural discrepancy between regular and deferred sensors that has been outstanding for many months which, it seems, will finally be [addressed](https://github.com/apache/airflow/pull/33718) in 2.11.x. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
nathadfield closed issue #43948: Wrong timeout value for ExternalTaskSensor running in deferrable mode URL: https://github.com/apache/airflow/issues/43948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
karakanb commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2511906398 Actually, testing this out, it does not work indeed across attempts, instead the timeout is only enforced within the same attempt across different reschedules. I think @kien-truong is right. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
karakanb commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2507405228 I think the document is correct with regards to what it says: it says "the `execution_timeout` and the `timeout` behave the same for `poke`", which means that they don't do what we'd expect them to do, and instead just set the timeout for the specific retry attempt instead. I agree with you that they should behave the same way. I think it'd be a relatively simple fix, but there needs to be a state that stores the attempts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
kien-truong commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2507333955 > When the user sets a timeout, the intention is "wait this long from the beginning of the first try Yeah, even in `poke` mode sensor does not respect this behavior, but use the current run start_date as the starting point, which I think is a bug in itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
karakanb commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2507302897 I looked into this a bit but it seems like there's a fundamental issue here, I'll try to explain below. The expected behavior would be to have a sensor that can run with retries, in case something fails during the sensor check, e.g. infra issues. The retries are not about the sensor not finding what it was supposed to, e.g. "the task is not there", but to recover from infra failures, e.g. the database being temporarily unavailable. This behavior works as expected with sensors in general. However, when combining retries on sensors with timeouts, that's where things start getting interesting: - When the user sets a timeout, the intention is "wait this long _from the beginning of the first try_", which is a very important factor that is also highlighted in the [Timeouts section of the docs](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#timeouts). This behavior seems to work correctly with `reschedule` mode thanks to the `task_reschedule` table that records the start timestamp for the first try. - However, when deferrable mode is used, the timeouts do not work with retries since there's no way to retrieve the start time of the first attempt of a task instance. It seems like the user would want the same behavior between deferred and non-deferred versions of the sensor for the timeouts with retries, but I couldn't find a way to solve it without adding a new table to airflow. is the original first start time information saved somewhere? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
potiuk commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2480387047 Assigned you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in deferrable mode [airflow]
venkateshwaracholan commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2479824699 @kien-truong I would like to work on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in Deferrable mode [airflow]
kien-truong commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2472308025 I can only contribute at weekend, so feel free to pick this up if anyone feels like it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Wrong timeout value for ExternalTaskSensor running in Deferrable mode [airflow]
boring-cyborg[bot] commented on issue #43948: URL: https://github.com/apache/airflow/issues/43948#issuecomment-2472296052 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
