tobixdev opened a new pull request, #20312: URL: https://github.com/apache/datafusion/pull/20312
## Which issue does this PR close? This is a PR based on https://github.com/apache/datafusion/pull/18552 that contains a basic implementation of an extension type registry. The driving use case is pretty-printing data frames with custom types. - Closes https://github.com/apache/datafusion/issues/18223. Ping @paleolimbot @adriangb if you're still interested. ### Most Important Changes to the Old PR - We no longer use the Logical Type, as there is no real conses on how DataFusion should allow "inline" references to extension types. As a consequence, the formatting query plans use case in the old PR no longer works. Extension types can only be used where DataFusion has a reference to a registry (e.g., DataFrame pretty-printing). @paleolimbot I've called it `DFExtensionType` instead of `BoundExtensionType` to avoid the need of explaining "bind". If you think there is merit in the other term, let me know. I think otherwise, this aligns with your proposal. - Added a more complex example with a parameterized type to demonstrate the entire ability of the API - No extension types are registered by default, users must opt-in ## Rationale for this change - Allow customized behavior based on extension type metadata. ## What changes are included in this PR? - Add an `ExtensionTypeRegistry` - Add `DFArrayFormatterFactory` which creates custom pretty-printers when formatting data frames. - Add an extension type registry to the `SessionState` / `SessionContext` - A Full Example of using the API - An implementation for the UUID canonical extension type ## Are these changes tested? - Yes, but only two end-to-end tests. - One for pretty-printing UUID values - One for pretty-printing in the example Happy to add more tests if this PR has a chance of being merged ## Are there any user-facing changes? Yes, the entire Extension Type API is new. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
