hudi-bot opened a new issue, #16558: URL: https://github.com/apache/hudi/issues/16558
With timestamp keygen you can have a partition column with timestamps, but then use the keygen so it will create partitions based on days so that all records that have a timestamp on 7-31-2024 will go to the same parititon even though the values in the partition column differ by hours and minutes etc. This causes a problem with partition pruning. lets say you query "select * from table where partition < 7-31-2024 at 7am and partition > 7-31-2024 at 6am ". Since the file structure has the partition of just 7-31-2024, that will be interpreted as 7-31-2024 at 12am. So the partition will be pruned from the search space. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-8037 - Type: Bug - Fix version(s): - 0.15.1 - 1.1.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
