Dear community, Happy to share Hudi community weekly update for 2020-02-16 ~ 2020-02-23 with updates on development, features and bugs.
Development [Query Improvements] A discussion to improve HUDI File Listing and Query Planning, and design doc is provided, it would reduce the number of file listing operations and improve query performance. [1] [Release] A discussion about releasing next Hudi version had reached a consensus, the next release is a minor release aims to tackle apache compliance issues. [2] [Index] A discussion to provide a record level indexing mechanisms for Hudi datasets, which is targeting for improving both write and read performance. [3] [Release] A discussion to the code freeze date for next release, the proposed time is 28th of Feb 23:59 PST, please chime in if you have any questions. [4] [Relocate dependency] Relocate spark-avro dependency by maven-shade-plugin has reached a consensus after a constructive conversation. [5] [1] https://lists.apache.org/thread.html/r4157f182595431d8aac98f5a4545b15ad513e468a44179e0e0aa6424%40%3Cdev.hudi.apache.org%3E [2] https://lists.apache.org/thread.html/r70c6741b7396d845d1eb79ddfed922287e9683ae399abd245497a8f8%40%3Cdev.hudi.apache.org%3E [3] https://lists.apache.org/thread.html/r5c8b33834ac14e25f6fdaf0b0cf64ac090e903f91926e584c9e23d85%40%3Cdev.hudi.apache.org%3E [4] https://lists.apache.org/thread.html/rfe3d4c9d89e9501b3d2993955a99d923081060d53a7b9d07c0843f7d%40%3Cdev.hudi.apache.org%3E [5] https://lists.apache.org/thread.html/r6bd85074eea3e807cde44ce9944c30966aa18c95e69b7bc9596d6398%40%3Cdev.hudi.apache.org%3E Features [Index] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4. [6] [Utilities] Add support for data types convertible to String in TimestampBasedKeyGenerator. [7] [6] https://jira.apache.org/jira/browse/HUDI-108 [7] https://jira.apache.org/jira/browse/HUDI-617 Bugs [Writer] Refactoring getter to avoid double extrametadata in json representation of HoodieCommitMetadata. [8] [8] https://jira.apache.org/jira/browse/HUDI-573 Best, Leesf
