Dear community, Nice to share Hudi community bi-weekly update for 2020-10-11 ~ 2020-10-25 with updates on features, bugfixs and tests.
======================================= Features [Flink Integration] Introduce HoodieFlinkEngineContext to hudi-flink-client [1] [Hive Integration] Add support for timestamp field in HiveSync [2] [Writer Core] IBM Cloud Object Storage Support [3] [Metrics] Added an API to force publish metrics and flush them [4] ======================================= Bugs [DeltaStreamer] Replace null by Option in Delta Streamer [5] [Index] Fix the support of hbase index partition path change [6] [Writer Core] Add better error messages when IOException occurs during log file reading [7] [Writer Core] Remove relocation of pattern for hbase dependencies and add shading of guava in hadoop, spark, and presto bundles [8] [Writer Core] Remove Hbase and htrace relocation from utilities bundle [9] [Writer Core] handle prefix filtering at directory level [10] [Writer Core] fixed NPE in CustomKeyGenerator [11] [DeltaStreamer] Properties File must be optional when running deltastreamer [12] ====================================== Tests [Test] Migrate HoodieTestUtils APIs to HoodieTestTable [13] [Test] Add unit test for testing compaction on replaced file groups [14] [Test] add test to check timestamp date decimal type write and read consistent [15] [Test] add more test for UpdateSchemaEvolution [16] [1] https://issues.apache.org/jira/browse/HUDI-1308 [2] https://issues.apache.org/jira/browse/HUDI-1302 [3] https://issues.apache.org/jira/browse/HUDI-1344 [4] https://issues.apache.org/jira/browse/HUDI-1326 [5] https://issues.apache.org/jira/browse/HUDI-791 [6] https://issues.apache.org/jira/browse/HUDI-1184 [7] https://issues.apache.org/jira/browse/HUDI-1298 [8] https://issues.apache.org/jira/browse/HUDI-1289 [9] https://issues.apache.org/jira/browse/HUDI-1345 [10] https://issues.apache.org/jira/browse/HUDI-1330 [11] https://issues.apache.org/jira/browse/HUDI-1200 [12] https://issues.apache.org/jira/browse/HUDI-1209 [13] https://issues.apache.org/jira/browse/HUDI-995 [14] https://issues.apache.org/jira/browse/HUDI-1304 [15] https://issues.apache.org/jira/browse/HUDI-307 [16] https://issues.apache.org/jira/browse/HUDI-284 Best, Leesf