vincbeck commented on code in PR #76497:
URL: https://github.com/apache/airflow/pull/76497#discussion_r4618560250
##########
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/utils/test_pod_manager.py:
##########
@@ -1346,12 +1352,38 @@ def test_extract_xcom_success(
mock_pod = MagicMock()
mock_client = MagicMock()
mock_client.peek_stderr.return_value = ""
- mock_client.read_all.return_value = xcom_json
+ mock_client.read_all.return_value = xcom_json.encode()
mock_kubernetes_stream.return_value = mock_client
ret = self.pod_manager.extract_xcom(pod=mock_pod)
assert ret == xcom_json
assert mock_exec_xcom_kill.call_count == 1
+ @mock.patch("kubernetes.stream.ws_client.create_websocket")
Review Comment:
This test relies a lot on `WSClient` internals (`create_websocket`, the
socket poll, the extra byte so it does not hang). A minor kubernetes client
bump could break it or make it hang. Would mocking `kubernetes_stream` and
asserting `binary=True` be enough?
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py:
##########
@@ -1049,14 +1049,17 @@ def extract_xcom_json(self, pod: V1Pod) -> str:
stderr=True,
tty=False,
_preload_content=False,
+ binary=True,
)
) as client:
self.log.info("Running command... %s", command)
client.run_forever()
if client.peek_stderr():
- stderr = client.read_stderr()
+ stderr = client.read_stderr().decode("utf-8", errors="replace")
self.log.error("stderr from command: %s", stderr)
- result = client.read_all()
+ raw_result = client.read_all()
Review Comment:
`read_all()` returns stdout and stderr together, so anything the sidecar
prints on stderr ends up in the XCom (and a stderr frame can still split a
character). Since you are changing this line, should we use `read_stdout()`
instead?
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/utils/pod_manager.py:
##########
@@ -1049,14 +1049,17 @@ def extract_xcom_json(self, pod: V1Pod) -> str:
stderr=True,
tty=False,
_preload_content=False,
+ binary=True,
)
) as client:
self.log.info("Running command... %s", command)
client.run_forever()
if client.peek_stderr():
- stderr = client.read_stderr()
+ stderr = client.read_stderr().decode("utf-8", errors="replace")
self.log.error("stderr from command: %s", stderr)
- result = client.read_all()
+ raw_result = client.read_all()
+ if raw_result is not None:
Review Comment:
nit: `read_all()` never returns `None` (it returns `b""` when there is no
output), this check is only there to keep `test_extract_xcom_none` passing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]