notfilippo commented on issue #11513: URL: https://github.com/apache/datafusion/issues/11513#issuecomment-2247428607
> But we actually have List(Uint8) inside arrow::Schema, so we can decode the type with List(UInt8). Therefore, we have UserDefinedType(Utf8) in the logical planning step, and List(Uint8) in the physical planning step Makes sense and I like this approach. The only thing I'm still not understanding is how a type source (like a TableProvider) would communicate to the logical layer that the `List(UInt8)` is actually a logical `Utf8`. > The most importance thing is that we could keep DFSchema mostly the same as it is now. But we introduce a bunch of new LogicalType and Trait for UserDefinedType to built around it. This would be tremendously beneficial to this proposal :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
