potiuk commented on issue #35474: URL: https://github.com/apache/airflow/issues/35474#issuecomment-1800106090
This is ilikely something we cannot do anything about you need to raise it to Kafka developers. The problem is that if you have C library that hangs and does not periodically check for signals we cannot do much. https://python.readthedocs.io/en/stable/library/signal.html > A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes. Possibly we could think about additional escalation level and sending sigkill in such cases after additional timeout. WDYT @Taragolis ? I think that should be possible as we are forking in local task job and we have a hearbeating process that is pure python that could potentiall send SIGKILL to the actual "task" process. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
