Dear community, Nice to share Hudi community bi-weekly updates for 2021-02-28 ~ 2021-03-14 with updates on features, bug fixes and tests.
======================================= Features [Flink Integration] Supports merge on read write mode for Flink writer [1] [Timeline] Configuration and metrics for the TimelineService [2] [Core] Add latency and freshness support [3] [Flink Integration] Supports snapshot read for Flink [4] [Core] Provide mechanism to read uncommitted data through InputFormat [5] [DFS] Support custom date format and fix unsupported exception in DatePartitionPathSelector [6] [Flink Integration] Streaming read for Flink MOR table [7] [Flink Integration] Row level delete for Flink sink[8] [Flink Integration] Avro schema inference for Flink SQL table [9] [Flink Integration] Support object storage for Flink writer [10] [1] https://issues.apache.org/jira/browse/HUDI-1632 [2] https://issues.apache.org/jira/browse/HUDI-1553 [3] https://issues.apache.org/jira/browse/HUDI-1587 [4] https://issues.apache.org/jira/browse/HUDI-1647 [5] https://issues.apache.org/jira/browse/HUDI-1646 [6] https://issues.apache.org/jira/browse/HUDI-1655 [7] https://issues.apache.org/jira/browse/HUDI-1663 [8] https://issues.apache.org/jira/browse/HUDI-1678 [9] https://issues.apache.org/jira/browse/HUDI-1664 [10] https://issues.apache.org/jira/browse/HUDI-1681 ======================================= Bugs [Core] Fix bug that Hudi will skip remaining log files if there is logFile with zero size in logFileList when merge on read[1] [Bundle] Fixing commons codec dependency in bundle jars [2] [Core] Do not delete older rollback instants as part of rollback [3] [Core] Re-bootstrap metadata table when un-synced instants have been archived. [4] [Flink Integration] Modify maker file path, which should start with the target base path [5] [Core] Support Builder Pattern To Build Table Properties For HoodieTableConfig [6] [Core] Excluding compaction and clustering instants from inflight rollback [7] [Core] Exclude clustering commits from getExtraMetadataFromLatest API [8] [DeltaStreamer] Fixing NPE with Parquet src in multi table delta streamer [9] [Hive Integration] Fix hive date type conversion for mor table [10] [Flink Integration] Replace scala.Tule2 to Pair in FlinkHoodieBloomIndex [11] [Core] Fix archival of requested replacecommit [12] [Core] keep updating current date for every batch [13] [GCS] Fixing input stream detection of GCS FileSystem [14] [1] https://issues.apache.org/jira/browse/HUDI-1583 [2] https://issues.apache.org/jira/browse/HUDI-1540 [3] https://issues.apache.org/jira/browse/HUDI-1644 [4] https://issues.apache.org/jira/browse/HUDI-1634 [5] https://issues.apache.org/jira/browse/HUDI-1584 [6] https://issues.apache.org/jira/browse/HUDI-1636 [7] https://issues.apache.org/jira/browse/HUDI-1660 [8] https://issues.apache.org/jira/browse/HUDI-1661 [9] https://issues.apache.org/jira/browse/HUDI-1618 [10] https://issues.apache.org/jira/browse/HUDI-1662 [11] https://issues.apache.org/jira/browse/HUDI-1673 [12] https://issues.apache.org/jira/browse/HUDI-1651 [13] https://issues.apache.org/jira/browse/HUDI-1685 [14] https://issues.apache.org/jira/browse/HUDI-1496 ======================================= Tests [Tests] Improvements to Hudi Test Suite [1] [1] https://issues.apache.org/jira/browse/HUDI-1635 Best, Leesf