darshil929 commented on issue #45930: URL: https://github.com/apache/airflow/issues/45930#issuecomment-4847409909
I looked into this on current main, since the thread trailed off on "try Airflow 3", and it still reproduces. I wrapped two workloads in the same TimeoutPosix/SIGALRM context the task runner applies around execute, both with a 1s execution_timeout. A plain time.sleep is killed right at 1s, but a blocking native query only fails once it returns, around 5.7s in my run. I used a long sqlite3 query for the native side since it behaves the same way as mysqlclient here. So the alarm fires on time, it just can't interrupt the native call until control returns to Python. To answer the question raised earlier in the thread: on main, execution_timeout is still handled in the task runner via SIGALRM, not in the supervisor (supervisor.py doesn't act on it), so the Airflow 3 execution rework didn't change this part. The general fix appears to be tracked already in [#53337](https://github.com/apache/airflow/issues/53337) (moving enforcement to the supervisor), with an earlier attempt in [#59657](https://github.com/apache/airflow/pull/59657) that was closed for inactivity. I'd be glad to take this on if it's helpful. Do you have a preference on the direction, reviving the supervisor-level approach or something narrower on the common-sql side such as a DB-side statement timeout? Happy to go whichever way you prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
