slice-soupam commented on code in PR #72008:
URL: https://github.com/apache/airflow/pull/72008#discussion_r3889881071
##########
airflow-core/src/airflow/dag_processing/manager.py:
##########
@@ -1658,7 +1721,7 @@ def _kill_timed_out_processors(self):
"dag_processing.processor_timeouts",
tags=prune_dict({"file_path": file_path_tag, "team_name":
team_name}),
)
- processor.kill(signal.SIGKILL)
+ processor.kill(signal.SIGKILL, wait=False)
Review Comment:
`processor` here is a `DagFileProcessorProcess`, which subclasses
`WatchedSubprocess` and doesn't override `kill()` — so `processor.kill()` runs
the shared `kill()` in `supervisor.py`. Each processor is also constructed with
`selector=self.selector` (the manager's one shared selector), so
`processor.selector` *is* the same object the manager uses everywhere else.
`kill(wait=True)` (the old default) doesn't just block — after sending the
signal it loops calling `_service_subprocess()`, which does
`self.selector.select(...)` and dispatches ready sockets. Since this PR added a
background `dag-processor-ipc` thread that continuously runs that same
servicing loop on that same shared selector, `kill(wait=True)` would make the
main thread act as a second concurrent consumer of it — racing the IPC thread
over events that may belong to *other*, still-running processors, and blocking
the main scheduling loop for up to `escalation_delay` (5s) in the process.
`wait=False` returns immediately after sending the signal, so the selector
stays owned exclusively by the IPC thread. The killed process's socket still
gets drained/closed normally on that thread's next iteration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
