nagasrisai opened a new pull request, #71811: URL: https://github.com/apache/airflow/pull/71811
Closes #64959 When `terminate_orphan_processes` kills a parsing subprocess, `kill()` returns as soon as the exit code is known, which can be before EOF was read from the subprocess sockets. Those sockets stay registered in the shared selector, so a later `select()` delivers stale events whose callbacks write to the log file handle that `close()` has already closed. The `ValueError: write to closed file` then propagates out of the parsing loop and crashes the whole DagProcessorJob. Two changes: * `DagFileProcessorProcess.close()` now unregisters and closes any sockets that never reported EOF before it closes the log file handle, so no stale selector events can fire afterwards. * `process_log_messages_from_subprocess()` drops leftover log lines when the target file handle is already closed instead of letting the `ValueError` escape, as a second layer of protection for the supervisor. Added a unit test on each side: one that verifies close() drains sockets a killed processor left registered, and one that feeds a log line into the forwarder after the log file was closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
