amoghrajesh commented on code in PR #45344:
URL: https://github.com/apache/airflow/pull/45344#discussion_r1900704060
##########
task_sdk/src/airflow/sdk/execution_time/supervisor.py:
##########
@@ -726,7 +727,12 @@ def _handle_request(self, msg: ToSupervisor, log:
FilteringBoundLogger):
resp = var_result.model_dump_json().encode()
elif isinstance(msg, GetXCom):
xcom = self.client.xcoms.get(msg.dag_id, msg.run_id, msg.task_id,
msg.key, msg.map_index)
- xcom_result = XComResult.from_xcom_response(xcom)
+ if isinstance(xcom, XComResponse):
+ xcom_result = XComResult.from_xcom_response(xcom)
+ else:
+ # Airflow 2.x just ignores the absence of an XCom and moves on
with a return value of None
+ # Hence making this resp with key as `key` and value as None,
so that the message is sent back to task runner.
+ xcom_result =
XComResult.from_xcom_response(XComResponse(key=msg.key, value=None))
Review Comment:
This branch indicates that the "xcom" is not found (The server responded
404).
Instead of complicating the client which is the sdk, I chose to send a
message to the task runner (it will receive it through get_message) with an
empty "value". If the value is empty, the `xcom_pull` in task runner will
return the `default` which is `None` anyways:
https://github.com/apache/airflow/blob/main/task_sdk/src/airflow/sdk/execution_time/task_runner.py#L198-L204
which is the behaviour I am intending.
##########
task_sdk/tests/execution_time/test_supervisor.py:
##########
@@ -762,14 +764,15 @@ def watched_subprocess(self, mocker):
)
@pytest.mark.parametrize(
- ["message", "expected_buffer", "client_attr_path", "method_arg",
"mock_response"],
+ ["message", "expected_buffer", "client_attr_path", "method_arg",
"mock_response", "decoded_buffer"],
Review Comment:
The `decoded_buffer` parameter overrides the comparison in
https://github.com/apache/airflow/pull/45344/files#diff-f55b9ba6be46de22b2838085bf1e0aa4193797c3ff7d0cdc4e06a626c96df1baR970-R971.
The idea is that when the supervisor a response to the buffer, if
`decoded_buffer` is present, it is used for comparison instead of
`mock_response`. This is logical because for cases like these when the client
responds 404 and we do not pass that all the way to task runner but transform
it to something else like in this case (client responded 404, but we sent
XcomResponse with value=None).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
