Re: [I] Add support for conversion of in memory tables to protobuf [datafusion-python]
timsaucer commented on issue #898: URL: https://github.com/apache/datafusion-python/issues/898#issuecomment-4215993051 @aditanase makes a very good point. Looking back on this a year later and it makes sense to me that we do not need to support these tables. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] Add support for conversion of in memory tables to protobuf [datafusion-python]
timsaucer closed issue #898: Add support for conversion of in memory tables to protobuf URL: https://github.com/apache/datafusion-python/issues/898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] Add support for conversion of in memory tables to protobuf [datafusion-python]
aditanase commented on issue #898: URL: https://github.com/apache/datafusion-python/issues/898#issuecomment-2829701511 What would be the expectation in that case? A MemTable resides on your heap, while CSV is potentially on disk. Would you expect that the data would be serialized as part of the plan? How would the execution work on the other side? I've seen some of this done in the substrait project: https://substrait.io/relations/logical_relations/#virtual-table I would imagine that one could write an extension codec that serializes the arrow table as well, but it would be extra work, not sure of the use case though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
