dhkim1920 commented on code in PR #73783:
URL: https://github.com/apache/airflow/pull/73783#discussion_r4326226964
##########
task-sdk/src/airflow/sdk/execution_time/comms.py:
##########
@@ -176,14 +180,44 @@ def _new_encoder() -> msgspec.msgpack.Encoder:
return msgspec.msgpack.Encoder(enc_hook=_msgpack_enc_hook)
+def _encode_large_ints(value: Any) -> Any:
+ if isinstance(value, int) and not -(2**63) <= value < 2**64:
+ return msgspec.msgpack.Ext(
+ MP_HUGEINT, value.to_bytes((value.bit_length() + 8) // 8, "big",
signed=True)
+ )
+ if isinstance(value, dict):
+ return {key: _encode_large_ints(item) for key, item in value.items()}
+ if isinstance(value, (list, tuple)):
+ return [_encode_large_ints(item) for item in value]
+ if isinstance(value, msgspec.Struct):
+ return msgspec.structs.replace(
+ value, **{name: _encode_large_ints(getattr(value, name)) for name
in value.__struct_fields__}
+ )
+ if isinstance(value, BaseModel):
+ return _encode_large_ints(value.model_dump(exclude_unset=True))
Review Comment:
I've simplified the fallback to recursively convert only the frame's body
and error payloads, preserving the outer frame and its metadata. The generic
msgspec.Struct and BaseModel traversal branches have been removed.
The tests now exercise the actual request-building and error-response paths,
including model_dump(), rather than injecting models directly into frames.
This scopes large-integer conversion to nested dictionaries, lists, and
tuples. model_dump() can still leave custom objects in Any-typed fields, so
this does not guarantee large-integer support inside arbitrary custom objects.
Nested container handling is still needed because oversized integers bypass
msgspec's enc_hook.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
