Dear community, Nice to share Hudi community bi-weekly update for 2020-10-25 ~ 2020-11-08 with updates on features, bugfixs and tests.
======================================= Features [Writer Core] Cleanup rollback files residing in .hoodie folder [1] [Hive Integration] Make hive synchronization supports hourly partition [2] [Writer Core] Use RateLimiter instead of sleep. Repartition WriteStatus to optimize Hbase index writes [3] [Writer Core] Refactor and relocate KeyGenerator to support more engines [4] [Hive Integration] RealtimeParquetInputFormat skip adding projection columns if there are no log files [4] [Writer Core] Add FileSystemView APIs to query pending clustering operations [5] ======================================= Bugs [Writer Core] Fix bug in HoodieAvroUtils.removeMetadataFields() method [6] ====================================== Tests [Test] Improvements to the hudi test suite for scalability and repeated testing. [7] [Test] Adding Delete support to test suite framework [8] [1] https://issues.apache.org/jira/browse/HUDI-1118 [2] https://issues.apache.org/jira/browse/HUDI-1274 [3] https://issues.apache.org/jira/browse/HUDI-316 [4] https://issues.apache.org/jira/browse/HUDI-912 [5] https://issues.apache.org/jira/browse/HUDI-1352 [6] https://issues.apache.org/jira/browse/HUDI-1375 [7] https://issues.apache.org/jira/browse/HUDI-1351 [8] https://issues.apache.org/jira/browse/HUDI-1338 Best, Leesf