bnspopi opened a new pull request, #77302: URL: https://github.com/apache/airflow/pull/77302
`CommsDecoder` writes a request and then takes the next frame on the socket as its response. It never compares the frame id with the request id. If an exception unwinds the exchange after the request was written, the response stays on the socket and the next request reads it as its own. `AirflowTaskTimeout` raised by the `execution_timeout` alarm can do this to any task that is waiting on an SDK call. The task can then get the previous `GetVariable`'s answer for a `GetConnection`, or fail to decode it. If the exception arrives part way through reading a frame, the framing itself is lost. That is what #77240 reports. This change does two things in `task-sdk/src/airflow/sdk/execution_time/comms.py`: - `send()` and `asend()` wait for the frame whose id matches the request. Responses to older, interrupted requests are discarded with a warning. A response with a newer id than the request raises, because it cannot happen with a correct supervisor. - `_read_frame` keeps the bytes of a partly read frame (and any file descriptors received with it), so the next read finishes that frame before starting a new one. The unsolicited startup message is still read with `_get_response()` and no id, so the task runner and Dag processor startup paths are unchanged. `TriggerCommsDecoder` already matches responses by id in its own reader loop and is not touched. Not covered: an exception raised while the request itself is being written can leave a partial request on the socket. That needs a change on the write side and is left for a follow-up. Tests: three cases for a request interrupted before the response is read, inside the length header and inside the body, plus one for a response with a newer id. I could not run the test suite locally (no Airflow dev environment on this machine), so this relies on CI. closes: #77240 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (Claude Code) Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
