bolkedebruin commented on issue #51016:
URL: https://github.com/apache/airflow/issues/51016#issuecomment-2952260346
I have looked into this. While we can fix serialization of primitives to do
only so (no derivatives), there is no easy way to to have generic Enum
serialization. Enums in python do not have an archetype, meaning that you
cannot do `x = Enum.IntEnum([1,2,3])` and will always end up with a new type
name (`c = Enum.IntEnum("Color", [1,2,3])` where c becomes of type 'Color').
That won't really scale for an allowed_list and isn't really intuitive I think
(correct me if I am wrong). This is an issue for other serialization libraries
as well (e.g. pydantic is struggling with the same issue).
The most 'beautiful' solution is to introspect the DAG and to find the
return type for a particular argument and see if the to-be deserialized value
is of the same type as the return type. Assuming the type has already been
loaded (e.g. literal string typing wouldn't be allowed) the security issue is
gone (we dont load arbitrary data). That's absolutely possible but will take
some more time to implement.
So for now I am proposing (see PR) to only serialize `pure` primitives and
keep the behavior for Enums as it is now until we can move to a better solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
