ethan-tyler commented on issue #20135: URL: https://github.com/apache/datafusion/issues/20135#issuecomment-3842682777
Thanks @alamb for getting this organized! I want to float a direction choice- modeling these as relation scoped system/hidden columns, rather than scalar functions. A column-like API handles join qualification naturally (`a.<meta>` vs `b.<meta>`), whereas a function like `filename()` has ambiguity issues. We can always add function syntax later as a rewrite. Coincidentally looks like SpiceAI was able to implement file scoped metadata columns (thanks @lukekim for sharing!) However, I think we should implement these as system/hidden columns and decide visibility + collision policy up front. If this direction resonates, I have some ideas but wanted to pause and see if folks agree. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
