Dear community, Nice to share Hudi community bi-weekly updates for 2022-03-28 ~ 2022-04-10 with updates on features, bug fixes.
======================================= Features [Core] Support Compaction Command Based on Call Procedure Command for Spark SQL [1] [Core] Implement Hudi AWS Glue sync [2] [Core] Add hudi-datahub-sync implementation [3] [Core] Implement async metadata indexing [4] [Core] Support full Schema evolution for Spark [5] [Flink] flink supports sync table information to aws glue [6] [Core] MVP implementation of BigQuerySyncTool [7] [1] https://issues.apache.org/jira/browse/HUDI-3538 [2] https://issues.apache.org/jira/browse/HUDI-2757 [3] https://issues.apache.org/jira/browse/HUDI-3536 [4] https://issues.apache.org/jira/browse/HUDI-3175 [5] https://issues.apache.org/jira/browse/HUDI-2429 [6] https://issues.apache.org/jira/browse/HUDI-3771 [7] https://issues.apache.org/jira/browse/HUDI-3357 ======================================= Bugs [Core] High performance costs of AvroSerizlizer in DataSource writing [1] [Flink] Flink bucket index bucketID bootstrap optimization [2] [Core] Fix the logic of reattempting pending rollback [3] [Core] Fix truncate hudi table's error [4] [Core] Fix drop table issue when sync to Hive [5] [Core] Fixing Column Stats Index record Merging sequence missing `columnName` [6] [Core] Fix drop partition issue when sync to hive [7] [Core] Removing dependency on "spark-avro" [8] [Core] Fix CTAS statment issue when sync to hive [9] [Flink] Fix flink bucket index bulk insert generates too many small files [10] [Core] Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy [11] [Core] fixed the per regression by enable vectorizeReader for parquet file [12] [Core] Improve HoodieSparkSqlWriter write performance [13] [Core] The MOR DELETE block breaks the event time sequence of CDC [14] [Core] fixed the bug that the cow table(contains decimalType) write by flink cannot be read by spark [15] [1] https://issues.apache.org/jira/browse/HUDI-3719 [2] https://issues.apache.org/jira/browse/HUDI-3539 [3] https://issues.apache.org/jira/browse/HUDI-3720 [4] https://issues.apache.org/jira/browse/HUDI-3722 [5] https://issues.apache.org/jira/browse/HUDI-2520 [6] https://issues.apache.org/jira/browse/HUDI-3731 [7] https://issues.apache.org/jira/browse/HUDI-2520 [8] https://issues.apache.org/jira/browse/HUDI-3549 [9] https://issues.apache.org/jira/browse/HUDI-2520 [10] https://issues.apache.org/jira/browse/HUDI-3741 [11] https://issues.apache.org/jira/browse/HUDI-3355 [12] https://issues.apache.org/jira/browse/HUDI-3729 [13] https://issues.apache.org/jira/browse/HUDI-2777 [14] https://issues.apache.org/jira/browse/HUDI-2752 [15] https://issues.apache.org/jira/browse/HUDI-3096 Best, Leesf