alamb commented on issue #1220: URL: https://github.com/apache/arrow-datafusion/issues/1220#issuecomment-957526700
Auto detecting common partitioning schemes seems like a good idea to me @Jimexist Something else I have been wondering is "how general do we want our partitioning to be" -- what @rdettai has implemented is the classic hive partitioning when partitioning by date looks like `/foo/date=2021-10-01` But there are other ways to partition data (e.g. IOx has its own way to partition data into individual files but the partition metadata is stored in some in-memory catalog) -- it would be pretty cool to re-use all the partitioning infrastructure (as well as, for example, add more sophisticated partition pruning) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
