paleolimbot commented on issue #18223:
URL: https://github.com/apache/datafusion/issues/18223#issuecomment-3453165634
Cool!
> If I want to define a custom string representation for a type, I need to
have access to that in the printing logic.
Luckily I think this one might be a fairly isolated problem...error messages
and explain plans are mostly where these show up and we can do a solidly OK job
at rendering extension types without a registry
(`arrow.fixed_size_tensor({stuff}) < storage >`). Not ideal but gets the point
across. Type equality is a bit more important I think but we can also do a
solidly OK job with this (see `PartialEq` and `Display` for the
`SerializedTypeView` in https://github.com/apache/datafusion/pull/18277). The
perfect solution would obviously dispatch dynamically for both of those.
> We have one problem here that ExtensionType is not dyn-compatible due to
the use of an associated constant and an associated type.
I also ran into this in https://github.com/apache/arrow-rs/pull/7398 ...it's
possibly useful for helping to fill in some of the boilerplate but I didn't
find it all that helpful for customizing the behaviour I was interested in
customizing.
> I got an error with unwind safety in the tests due to the dynamic dispatch
(see diff).
I think I solved that one with `pub trait DynExtensionType: Debug +
RefUnwindSafe { ... }` but I'm not an expert here and maybe that's a separate
issue.
> But this would probably be the approach of least resistance (from the API
breakage perspective). Any thoughts on that ?
I have historically had bad luck guessing what will and won't meet
resistance in arrow-rs and have found it more enjoyable to focus on solutions
elsewhere. For example, much of what we're trying to do here can be done with
logical plan transformations or a separate UDF framework (which is what we do
in SedonaDB).
> I am currently exploring directly integrating the dyn ExtensionTypeThingey
in the expressions and plans. If we decide to go with another option (pass
registry or TypeExtensions) this could be an interessting approach!
This sounds hard but I look forward to seeing what you come up with!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
