alamb commented on PR #27210: URL: https://github.com/apache/datafusion/pull/27210#issuecomment-6989651433
I wonder if we could add some sort of general purpose API to the TableProvider to attach "user defined" extensions like there is to pass information down to a file scan / opener: https://github.com/apache/datafusion/blob/407bef62ca965815af27624564937aae50c3578a/datafusion/datasource/src/mod.rs#L152-L155 We could then perhaps use the existing planner hooks to add the user defined extensions to the table provider / TableScan ? We would still have to expand TableScan to implement PartialEq, as well as sort out the async catalog access. However, the upside would be that the time travel specific parts (`AsOfVersion`, the selector planner, etc) could live in an extension rather than the core, which I think is similar to what @alexanderbianchi is suggesting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
