Dear community, Happy New Year everyone. Nice to share Hudi community bi-weekly updates for 2020-12-20 ~ 2021-01-03 with updates on features, bug fixes and tests.
======================================= Features [Core] Adding DefaultHoodieRecordPayload to honor ordering with combineAndGetUpdateValue [1] [Client] Add base implementation for hudi java client [2] [Core] Implement simple clustering strategies to create ClusteringPlan and to run the plan [3] [Core] Support bulk insert v2 with Spark 3.0.0 [4] [Core] Block updates and replace on file groups in clustering [5] [Core] Support Partition level delete API in HUDI [6] [Core] Upgrade Flink version to 1.12.0 [7] [Client] Support delete in hudi-java-client [8] [1] https://issues.apache.org/jira/browse/HUDI-115 [2] https://issues.apache.org/jira/browse/HUDI-1419 [3] https://issues.apache.org/jira/browse/HUDI-1075 [4] https://issues.apache.org/jira/browse/HUDI-1451 [5] https://issues.apache.org/jira/browse/HUDI-1354 [6] https://issues.apache.org/jira/browse/HUDI-1350 [7] https://issues.apache.org/jira/browse/HUDI-1495 [8] https://issues.apache.org/jira/browse/HUDI-1423 ======================================= Bugs [QuickStart] Make QuickStartUtils generate deletes according to specific ts [1] [Core] Fix Deletes issued without any prior commits exception [2] [Bootstrap] Fix null pointer exception when reading updated written bootstrap table [3] [Core] Incremental Query should work even when there are partitions that have no incremental changes [4] [Core] Align insert file size for reducing IO [5] [Hive Integration] Escape the partition value in HiveSyncTool [6] [Core] Modify GenericRecordFullPayloadGenerator to generate valid timestamps [7] [Core] Fixed schema compatibility check for fields [8] [Core] fix incorrect log file path in HoodieWriteStat [9] [1] https://issues.apache.org/jira/browse/HUDI-1471 [2] https://issues.apache.org/jira/browse/HUDI-1485 [3] https://issues.apache.org/jira/browse/HUDI-1489 [4] https://issues.apache.org/jira/browse/HUDI-1490 [5] https://issues.apache.org/jira/browse/HUDI-1398 [6] https://issues.apache.org/jira/browse/HUDI-1484 [7] https://issues.apache.org/jira/browse/HUDI-1147 [8] https://issues.apache.org/jira/browse/HUDI-1493 [9] https://issues.apache.org/jira/browse/HUDI-1434 ======================================= Tests [Tests] Fix Test Case Failure in TestHBaseIndex [1] [Tests] fix unit test testCopyOnWriteStorage random failed [2] [Tests] Adding support for validating entire dataset and long running tests in test suite framework [3] [Tests] add structured streaming and delta streamer clustering unit test [4] [Tests] Add additional unit tests to TestHBaseIndex [5] [Tests] Set up flink client unit test infra [6] [1] https://issues.apache.org/jira/browse/HUDI-1488 [2] https://issues.apache.org/jira/browse/HUDI-1487 [3] https://issues.apache.org/jira/browse/HUDI-1331 [4] https://issues.apache.org/jira/browse/HUDI-1481 [5] https://issues.apache.org/jira/browse/HUDI-1474 [6] https://issues.apache.org/jira/browse/HUDI-1418 Best, Leesf