potiuk commented on code in PR #44465:
URL: https://github.com/apache/airflow/pull/44465#discussion_r1863576653
##########
task_sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -403,12 +403,57 @@ def _send_startup_message(self, ti: TaskInstance, path:
str | os.PathLike[str],
self.stdin.write(msg.model_dump_json().encode())
self.stdin.write(b"\n")
- def kill(self, signal: signal.Signals = signal.SIGINT):
+ def kill(
+ self,
+ signal_to_send: signal.Signals = signal.SIGINT,
+ escalation_delay: float = 5.0,
+ force: bool = False,
+ ):
+ """
+ Attempt to terminate the subprocess with a given signal.
+
+ If the process does not exit within `escalation_delay` seconds,
escalate to SIGTERM and eventually SIGKILL if necessary.
+
+ :param signal_to_send: The signal to send initially (default is
SIGINT).
+ :param escalation_delay: Time in seconds to wait before escalating to
a stronger signal.
+ :param force: If True, send the signal immediately without escalation.
+ """
if self._exit_code is not None:
return
- with suppress(ProcessLookupError):
- os.kill(self.pid, signal)
+ if not force:
+ with suppress(ProcessLookupError):
+ self._process.send_signal(signal_to_send)
+ return
+
+ # Escalation sequence: SIGINT -> SIGTERM -> SIGKILL
+ escalation_path = [signal.SIGINT, signal.SIGTERM, signal.SIGKILL]
+ if signal_to_send in escalation_path:
+ # Start from `initial_signal`
+ escalation_path =
escalation_path[escalation_path.index(signal_to_send) :]
+
+ for sig in escalation_path:
+ try:
+ self._process.send_signal(sig)
Review Comment:
Not mentioning fork that also deoes not **really** work well on MacOS
(because on MacOS system libraries might start threads which does not play well
with threads https://github.com/python/cpython/issues/77906), so if we would
like to have explicit support for MacOS (and later Windows) for "workers" - we
will have to make some serious ifs
BTW. In Python 3.14 there are some changes coming that **might** finally
result in a possibility of using fork() in production for MacOS
https://docs.python.org/3/library/multiprocessing.html#multiprocessing-start-methods
- maybe we should already add "get_context()" and "set_start_method" to
explicity set "fork" on systems that support it (minus MacOS :) - as this will
be a breaking change in Python 3.14
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
