alamb commented on issue #20135: URL: https://github.com/apache/datafusion/issues/20135#issuecomment-3843212671
> FWIW, we implemented a version of this in ListingTable [spiceai#74](https://github.com/spiceai/datafusion/pull/74) Thanks @lukekim -- looks like @phillipleblanc implemented the metadata columns similarly to the partition columns (at the ListingTable level) which is a neat idea. It does seem like there are basically two approaches: 1. metadata columns (that are treated as a column on the releation) 2. Functions that are expanded below It seems like all the cool kids (SpiceAI, Spark, etc) are opting for the metadata column approach 🤔 It looks like snowflake has some sort of column / variable craziness (`METADATA$FILENAME`) https://docs.snowflake.com/en/user-guide/querying-metadata#example-3-loading-metadata-columns-into-a-table -- but that is all in the context of a load, so maybe there aren't multiple tables to worry about 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
