amoghrajesh commented on code in PR #49173:
URL: https://github.com/apache/airflow/pull/49173#discussion_r2041431789
##########
providers/common/io/src/airflow/providers/common/io/xcom/backend.py:
##########
@@ -160,9 +160,16 @@ def deserialize_value(result) -> Any:
Compression is inferred from the file extension.
"""
- data = BaseXCom.deserialize_value(result)
+ base_xcom_deser_result = BaseXCom.deserialize_value(result)
+
+ # When XComObjectStorageBackend is used, xcom value will be serialized
using json.dumps
+ # likely, we need to deserialize it using json.loads
+ try:
+ data = json.loads(base_xcom_deser_result, cls=XComDecoder)
+ except (TypeError, ValueError):
+ data = base_xcom_deser_result
Review Comment:
Ok i see whats happening here, its cos of
https://github.com/apache/airflow/blob/main/providers/common/io/src/airflow/providers/common/io/xcom/backend.py#L131-L138
Instead of doing this, which was done earlier because of the
deserialize_value accepting only xcom values, we should now be able to do this
directly via the BaseXcom.deserialize_value.
So instead a better approach would be to store it in the way we store with
xcom backend isn;t enabled, that is, in here:
https://github.com/apache/airflow/blob/main/providers/common/io/src/airflow/providers/common/io/xcom/backend.py#L121-L138
do not use json.dumps, instead use the BaseXcom.serialize_value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
