ColtenOuO opened a new pull request, #71962: URL: https://github.com/apache/airflow/pull/71962
closes: #49099 ## Summary `get_json_error()` only wraps 400-599 responses in the picklable `ServerResponseError` -- it needs a custom `__reduce__` because a plain `httpx.HTTPStatusError` requires keyword-only `request`/`response` arguments that pickle's default reconstruction can't supply (see #47873, fixed for that path by #48517). A 3xx response -- e.g. a corporate proxy in front of the API server returning a redirect -- falls outside that range, so `httpx.Response.raise_for_status()` raises the bare, unpicklable `httpx.HTTPStatusError` instead. That crashes the scheduler once the exception crosses a multiprocessing boundary: `LocalExecutor.result_queue` is a `multiprocessing.Queue`, and unpickling the exception on the scheduler side fails with the exact "missing keyword-only arguments" `TypeError` #47873 first reported -- just triggered by a redirect instead of a JSON error body. Reproduced and confirmed with a standalone script -- the same failure mode as #47873's crash log. ## Change Added `_raise_for_status_picklable()` in `task-sdk/src/airflow/sdk/api/client.py`, used by both `raise_on_4xx_5xx` and `raise_on_4xx_5xx_with_note`: anything `raise_for_status()` raises outside the 400-599 range now gets wrapped in a `ServerResponseError` (picklable) before propagating. Errors inside 400-599 that `from_response()` couldn't extract a JSON detail from are left exactly as before -- this only closes the gap for status codes `get_json_error()` never attempted to wrap in the first place. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes -- Claude Code (Sonnet 5) for writing test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
